2019-12-09T09:06:35.708 INFO:root:teuthology version: 1.0.0-8b630f9 2019-12-09T09:06:35.708 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.KBiXLB.tmp -- --verbose --lock --description rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} --name ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi --block --owner scheduled_ofriedma@teuthology --archive /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584223 2019-12-09T09:06:35.757 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T09:06:35.853 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584223 branch: wip-ofriedma-dynamic-disable description: rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} email: ofriedma@redhat.com first_in_suite: false job_id: '4584223' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos overrides: admin_socket: branch: wip-ofriedma-dynamic-disable ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 1351c819e1a9518cde4c8b12d645782009028360 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: notcmalloc sha1: 1351c819e1a9518cde4c8b12d645782009028360 rgw: compression type: random ec-data-pool: true frontend: civetweb storage classes: LUKEWARM, FROZEN s3tests: slow_backend: true thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-ofriedma-dynamic-disable sha1: 1351c819e1a9518cde4c8b12d645782009028360 owner: scheduled_ofriedma@teuthology priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 1351c819e1a9518cde4c8b12d645782009028360 suite: rgw suite_branch: wip-ofriedma-dynamic-disable suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 1351c819e1a9518cde4c8b12d645782009028360 tasks: - install: flavor: notcmalloc - ceph: null - openssl_keys: null - rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - rgw/test_rgw_gc_log.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - s3tests-java: client.0: force-branch: master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: force-branch: ceph-master rgw_server: client.0 - swift: client.0: rgw_server: client.0 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.13052 2019-12-09T09:06:35.853 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa; will attempt to use it 2019-12-09T09:06:35.854 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks 2019-12-09T09:06:35.854 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-12-09T09:06:35.856 INFO:teuthology.task.internal:Checking packages... 2019-12-09T09:06:35.876 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '1351c819e1a9518cde4c8b12d645782009028360' 2019-12-09T09:06:35.877 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-09T09:06:35.877 INFO:teuthology.packaging:ref: None 2019-12-09T09:06:35.877 INFO:teuthology.packaging:tag: None 2019-12-09T09:06:35.877 INFO:teuthology.packaging:branch: wip-ofriedma-dynamic-disable 2019-12-09T09:06:35.877 INFO:teuthology.packaging:sha1: 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:06:35.877 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=wip-ofriedma-dynamic-disable 2019-12-09T09:06:36.037 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7900.g1351c81 2019-12-09T09:06:36.038 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-12-09T09:06:36.047 INFO:teuthology.task.internal:no buildpackages task found 2019-12-09T09:06:36.048 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-12-09T09:06:36.061 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-12-09T09:06:36.062 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T09:06:36.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:06:46.145 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:06:56.161 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:07:06.175 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:07:16.193 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:07:26.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:07:36.224 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:07:46.242 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:07:56.257 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:08:06.278 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:08:16.294 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:08:26.309 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:08:36.346 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:08:46.362 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:08:56.377 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:09:06.396 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:09:16.414 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:09:26.433 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:09:36.456 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 1)... 2019-12-09T09:09:46.476 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:09:56.496 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:10:06.512 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:10:16.528 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:10:26.553 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:10:36.570 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:10:46.585 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:10:56.611 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:11:06.628 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:11:16.645 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:11:26.660 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T09:11:36.681 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 3)... 2019-12-09T09:11:46.698 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_ofriedma@teuthology', 'description': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584223', 'machine_type': 'smithi'} 2019-12-09T09:11:46.715 DEBUG:teuthology.lock.ops:locked ubuntu@smithi033.front.sepia.ceph.com, ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:11:46.808 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T09:11:47.045 INFO:teuthology.provision.fog.smithi033:Scheduling deploy of centos 7.6 2019-12-09T09:11:47.112 INFO:teuthology.provision.fog.smithi186:Scheduling deploy of centos 7.6 2019-12-09T09:11:47.485 INFO:teuthology.orchestra.console:Power off smithi033 2019-12-09T09:11:47.486 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T09:11:47.527 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:11:47.556 INFO:teuthology.orchestra.console:Power off smithi186 2019-12-09T09:11:47.556 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T09:11:47.581 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:11:47.678 INFO:teuthology.orchestra.console:Power off for smithi033 completed 2019-12-09T09:11:47.711 INFO:teuthology.orchestra.console:Power off for smithi186 completed 2019-12-09T09:11:47.779 INFO:teuthology.orchestra.console:Power on smithi033 2019-12-09T09:11:47.780 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T09:11:47.808 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:11:47.825 INFO:teuthology.orchestra.console:Power on smithi186 2019-12-09T09:11:47.828 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T09:11:47.854 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:11:51.848 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:11:51.880 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi186.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:11:52.096 INFO:teuthology.orchestra.console:Power on for smithi186 completed 2019-12-09T09:11:52.197 INFO:teuthology.provision.fog.smithi186:Waiting for deploy to finish 2019-12-09T09:11:55.964 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T09:11:56.177 INFO:teuthology.orchestra.console:Power on for smithi033 completed 2019-12-09T09:11:56.278 INFO:teuthology.provision.fog.smithi033:Waiting for deploy to finish 2019-12-09T09:14:23.372 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:14:23.470 ERROR:teuthology.orchestra.connection:Error connecting to smithi186.front.sepia.ceph.com Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/connection.py", line 106, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 437, in connect passphrase, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 749, in _auth raise saved_exception AuthenticationException: Authentication failed. 2019-12-09T09:14:27.372 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:14:36.426 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:14:45.484 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:14:54.542 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:15:03.593 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:15:12.653 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:15:21.710 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:15:30.473 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:15:59.145 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:15:59.391 INFO:teuthology.orchestra.run.smithi033:> hostname 2019-12-09T09:15:59.875 INFO:teuthology.orchestra.run.smithi033.stdout:smithi068 2019-12-09T09:15:59.876 INFO:teuthology.orchestra.run.smithi033:> grep smithi068 /etc/hosts 2019-12-09T09:15:59.924 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068 2019-12-09T09:15:59.925 INFO:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2019-12-09T09:16:00.023 INFO:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi068/smithi033/g' /etc/hosts 2019-12-09T09:16:00.085 INFO:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/172.21.15.68/172.21.15.33/g' /etc/hosts 2019-12-09T09:16:00.154 INFO:teuthology.orchestra.run.smithi033:> sudo sed -i -e 's/smithi068/smithi033/g' /etc/hostname 2019-12-09T09:16:00.210 INFO:teuthology.orchestra.run.smithi033:> sudo hostname smithi033 2019-12-09T09:16:00.269 INFO:teuthology.provision.fog.smithi033:Deploy complete! 2019-12-09T09:16:00.273 DEBUG:teuthology.parallel:result is None 2019-12-09T09:16:36.476 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:16:36.681 INFO:teuthology.orchestra.run.smithi186:> hostname 2019-12-09T09:16:37.233 INFO:teuthology.orchestra.run.smithi186.stdout:smithi186 2019-12-09T09:16:37.234 INFO:teuthology.orchestra.run.smithi186:> grep smithi186 /etc/hosts 2019-12-09T09:16:37.309 INFO:teuthology.orchestra.run.smithi186.stdout:172.21.15.186 smithi186.front.sepia.ceph.com smithi186 2019-12-09T09:16:37.309 INFO:teuthology.orchestra.run.smithi186:> sudo hostname smithi186 2019-12-09T09:16:37.455 INFO:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/smithi186/smithi186/g' /etc/hosts 2019-12-09T09:16:37.565 INFO:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/172.21.15.186/172.21.15.186/g' /etc/hosts 2019-12-09T09:16:37.652 INFO:teuthology.orchestra.run.smithi186:> sudo sed -i -e 's/smithi186/smithi186/g' /etc/hostname 2019-12-09T09:16:37.725 INFO:teuthology.orchestra.run.smithi186:> sudo hostname smithi186 2019-12-09T09:16:37.790 INFO:teuthology.provision.fog.smithi186:Deploy complete! 2019-12-09T09:16:37.792 DEBUG:teuthology.parallel:result is None 2019-12-09T09:16:37.969 INFO:teuthology.lock.ops:Checking smithi033.front.sepia.ceph.com 2019-12-09T09:16:37.970 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T09:16:38.905 INFO:teuthology.lock.ops:Checking smithi186.front.sepia.ceph.com 2019-12-09T09:16:38.905 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T09:16:38.938 INFO:teuthology.lock.ops:Updating [smithi033.front.sepia.ceph.com]: set os type and version on server 2019-12-09T09:16:38.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:16:38.939 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:16:39.037 INFO:teuthology.orchestra.run.smithi033:> true 2019-12-09T09:16:39.447 INFO:teuthology.orchestra.run.smithi033:> uname -m 2019-12-09T09:16:39.514 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2019-12-09T09:16:39.514 INFO:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2019-12-09T09:16:39.584 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Linux" 2019-12-09T09:16:39.584 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="7 (Core)" 2019-12-09T09:16:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2019-12-09T09:16:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:16:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="7" 2019-12-09T09:16:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:16:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2019-12-09T09:16:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:16:39.585 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:16:39.586 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:16:39.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:39.586 INFO:teuthology.orchestra.run.smithi033.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:16:39.586 INFO:teuthology.orchestra.run.smithi033.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:16:39.586 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:16:39.586 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:16:39.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:39.587 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2019-12-09T09:16:39.605 INFO:teuthology.lock.ops:Updating [smithi186.front.sepia.ceph.com]: set os type and version on server 2019-12-09T09:16:39.605 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:16:39.606 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:16:39.696 INFO:teuthology.orchestra.run.smithi186:> true 2019-12-09T09:16:40.069 INFO:teuthology.orchestra.run.smithi186:> uname -m 2019-12-09T09:16:40.138 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2019-12-09T09:16:40.138 INFO:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2019-12-09T09:16:40.204 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Linux" 2019-12-09T09:16:40.205 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="7 (Core)" 2019-12-09T09:16:40.205 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2019-12-09T09:16:40.205 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:16:40.205 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="7" 2019-12-09T09:16:40.205 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:16:40.206 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2019-12-09T09:16:40.206 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:16:40.206 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:16:40.206 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:16:40.206 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:16:40.207 INFO:teuthology.orchestra.run.smithi186.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:16:40.207 INFO:teuthology.orchestra.run.smithi186.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:16:40.207 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:16:40.207 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:16:40.208 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:16:40.208 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2019-12-09T09:16:40.229 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2019-12-09T09:16:40.261 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi033.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOi70/1lLZEdHzIrJ9izBFnq+89y3RD78C2AkrMEqStAtUlu1ZyS34ityJlJ7OzmUTbcl/I52cbRPvHdIeQjgbRBcMo2EF5pM+3+iW+H7vdrAdVGAVLE2pzUG9cnCKCmD7jTtYUJ2NJDCkVU9Mhz59ZvkT7ZWz33fVVhy2JQD477B5h3PyNqgELosKAdxRYkViX4aIh95ibVLoq0ipkBppbf494GelyI9JmpXHfV4EtrFr0+vKah6BFb3dzxJu5e1igDGHg16udXkqwKXrs3TQJkiZeI2+UpgJR3TayL6v2J4hF5TGysDlTNNJEEcyt5fnIr6CNmEr2ziq10Bm1jHT smithi186.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDY0LwYYJs/HJCINnWA3ZlN0URRZ57HW3F2VRemQKoDvkYDvSj4fJznS8yJYxdUUJGL5Ow0kMvZqYg3mg1A+xSMYY5ivzY6th6AUpb9vxl4IwNNN0Tbsg3vE72VyyIqKU2OPK/cuwDkj6zxIJk6VKSfLTOCQnkUALCNcQBE5R4i4g8njGnscjpE3/IxN9awxjzhmIl2Tw4rDmuTnHNH6sbkdwg57Hct4OBl9WHTlITLbrYaCUB6LYYYZROAKvC7Y8WYFyzDBebaVLcHXnwBI74RlntKDPSDHREWR+7ZvKS69+G6COzUAfFZ3W81YiVqWlkhlAZM3QOL9ehMFpiCjLlZ 2019-12-09T09:16:40.261 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T09:16:40.305 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-12-09T09:16:40.310 INFO:teuthology.task.internal:Saving configuration 2019-12-09T09:16:40.340 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-12-09T09:16:40.346 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-12-09T09:16:40.365 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 09:11:46.705844', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi033.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGynEaT2maL3zqgvWo2V1s4P6bd6xgwOKcOfD5GhE7bCH2gsbmsPW/ajSM73BLNj24HvJWXWWIV4LQm7JyXnrHE=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584223'} 2019-12-09T09:16:40.381 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 09:11:46.705864', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi186.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLQASSJ+vnzKTd+Ybe5XRbDLje31ucUS2MVF13IksavSO3D2tQIqM5ZGTZrERo4QOvb4ULY7xorRFq4Cx/cpyng=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584223'} 2019-12-09T09:16:40.381 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-12-09T09:16:40.386 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T09:16:40.387 INFO:teuthology.task.internal:roles: ubuntu@smithi186.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T09:16:40.387 INFO:teuthology.run_tasks:Running task console_log... 2019-12-09T09:16:40.471 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T09:16:40.472 INFO:teuthology.run_tasks:Running task internal.connect... 2019-12-09T09:16:40.504 INFO:teuthology.task.internal:Opening connections... 2019-12-09T09:16:40.504 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:16:40.505 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:16:40.607 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:16:40.608 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:16:40.705 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-12-09T09:16:40.715 INFO:teuthology.orchestra.run.smithi033:> uname -m 2019-12-09T09:16:40.964 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2019-12-09T09:16:40.964 INFO:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2019-12-09T09:16:41.040 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Linux" 2019-12-09T09:16:41.040 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="7 (Core)" 2019-12-09T09:16:41.040 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2019-12-09T09:16:41.041 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:16:41.041 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="7" 2019-12-09T09:16:41.041 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:16:41.041 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2019-12-09T09:16:41.041 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:16:41.041 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:16:41.042 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:16:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:41.042 INFO:teuthology.orchestra.run.smithi033.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:16:41.042 INFO:teuthology.orchestra.run.smithi033.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:16:41.042 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:16:41.042 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:16:41.042 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:41.043 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2019-12-09T09:16:41.059 INFO:teuthology.orchestra.run.smithi186:> uname -m 2019-12-09T09:16:41.180 INFO:teuthology.orchestra.run.smithi186.stdout:x86_64 2019-12-09T09:16:41.181 INFO:teuthology.orchestra.run.smithi186:> cat /etc/os-release 2019-12-09T09:16:41.264 INFO:teuthology.orchestra.run.smithi186.stdout:NAME="CentOS Linux" 2019-12-09T09:16:41.265 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION="7 (Core)" 2019-12-09T09:16:41.265 INFO:teuthology.orchestra.run.smithi186.stdout:ID="centos" 2019-12-09T09:16:41.265 INFO:teuthology.orchestra.run.smithi186.stdout:ID_LIKE="rhel fedora" 2019-12-09T09:16:41.265 INFO:teuthology.orchestra.run.smithi186.stdout:VERSION_ID="7" 2019-12-09T09:16:41.265 INFO:teuthology.orchestra.run.smithi186.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T09:16:41.265 INFO:teuthology.orchestra.run.smithi186.stdout:ANSI_COLOR="0;31" 2019-12-09T09:16:41.266 INFO:teuthology.orchestra.run.smithi186.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T09:16:41.266 INFO:teuthology.orchestra.run.smithi186.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T09:16:41.266 INFO:teuthology.orchestra.run.smithi186.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T09:16:41.266 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:16:41.266 INFO:teuthology.orchestra.run.smithi186.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T09:16:41.266 INFO:teuthology.orchestra.run.smithi186.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T09:16:41.266 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T09:16:41.267 INFO:teuthology.orchestra.run.smithi186.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T09:16:41.267 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:16:41.267 INFO:teuthology.lock.ops:Updating smithi186.front.sepia.ceph.com on lock server 2019-12-09T09:16:41.309 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-12-09T09:16:41.323 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-12-09T09:16:41.332 INFO:teuthology.task.internal:Checking for old test directory... 2019-12-09T09:16:41.333 INFO:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2019-12-09T09:16:41.340 INFO:teuthology.orchestra.run.smithi186:> test '!' -e /home/ubuntu/cephtest 2019-12-09T09:16:41.428 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-12-09T09:16:41.445 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-12-09T09:16:41.446 INFO:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2019-12-09T09:16:41.454 INFO:teuthology.orchestra.run.smithi186:> test -z $(ls -A /var/lib/ceph) 2019-12-09T09:16:41.551 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-12-09T09:16:41.594 INFO:teuthology.run_tasks:Running task kernel... 2019-12-09T09:16:41.612 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-12-09T09:16:41.612 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-12-09T09:16:41.613 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:16:41.613 INFO:teuthology.orchestra.run.smithi033:> sudo package-cleanup -y --oldkernels 2019-12-09T09:16:42.491 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:16:43.538 INFO:teuthology.orchestra.run.smithi033.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:43.539 INFO:teuthology.orchestra.run.smithi033.stdout:Trying other mirror. 2019-12-09T09:16:43.539 INFO:teuthology.orchestra.run.smithi033.stdout:To address this issue please refer to the below wiki article 2019-12-09T09:16:43.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:43.539 INFO:teuthology.orchestra.run.smithi033.stdout:https://wiki.centos.org/yum-errors 2019-12-09T09:16:43.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:43.539 INFO:teuthology.orchestra.run.smithi033.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T09:16:43.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:43.701 INFO:teuthology.orchestra.run.smithi033.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:43.701 INFO:teuthology.orchestra.run.smithi033.stdout:Trying other mirror. 2019-12-09T09:16:43.793 INFO:teuthology.orchestra.run.smithi033.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:43.793 INFO:teuthology.orchestra.run.smithi033.stdout:Trying other mirror. 2019-12-09T09:16:43.937 INFO:teuthology.orchestra.run.smithi033.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:43.937 INFO:teuthology.orchestra.run.smithi033.stdout:Trying other mirror. 2019-12-09T09:16:44.089 INFO:teuthology.orchestra.run.smithi033.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:44.089 INFO:teuthology.orchestra.run.smithi033.stdout:Trying other mirror. 2019-12-09T09:16:44.685 INFO:teuthology.orchestra.run.smithi033.stdout:No old kernels to remove 2019-12-09T09:16:44.705 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:16:44.705 INFO:teuthology.orchestra.run.smithi186:> sudo package-cleanup -y --oldkernels 2019-12-09T09:16:45.569 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:16:46.458 INFO:teuthology.orchestra.run.smithi186.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:46.458 INFO:teuthology.orchestra.run.smithi186.stdout:Trying other mirror. 2019-12-09T09:16:46.458 INFO:teuthology.orchestra.run.smithi186.stdout:To address this issue please refer to the below wiki article 2019-12-09T09:16:46.459 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:16:46.459 INFO:teuthology.orchestra.run.smithi186.stdout:https://wiki.centos.org/yum-errors 2019-12-09T09:16:46.459 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:16:46.459 INFO:teuthology.orchestra.run.smithi186.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T09:16:46.459 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:16:46.605 INFO:teuthology.orchestra.run.smithi186.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:46.605 INFO:teuthology.orchestra.run.smithi186.stdout:Trying other mirror. 2019-12-09T09:16:46.698 INFO:teuthology.orchestra.run.smithi186.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:46.698 INFO:teuthology.orchestra.run.smithi186.stdout:Trying other mirror. 2019-12-09T09:16:46.840 INFO:teuthology.orchestra.run.smithi186.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:46.840 INFO:teuthology.orchestra.run.smithi186.stdout:Trying other mirror. 2019-12-09T09:16:46.993 INFO:teuthology.orchestra.run.smithi186.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T09:16:46.993 INFO:teuthology.orchestra.run.smithi186.stdout:Trying other mirror. 2019-12-09T09:16:47.544 INFO:teuthology.orchestra.run.smithi186.stdout:No old kernels to remove 2019-12-09T09:16:47.558 INFO:teuthology.orchestra.run.smithi033:> uname -r 2019-12-09T09:16:47.667 INFO:teuthology.orchestra.run.smithi033.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:16:47.668 INFO:teuthology.task.kernel:Running kernel on smithi033: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:16:47.668 INFO:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2019-12-09T09:16:47.940 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:16:48.012 INFO:teuthology.orchestra.run.smithi033.stdout:Determining fastest mirrors 2019-12-09T09:16:48.148 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:16:48.151 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T09:16:48.151 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:16:48.151 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:16:49.744 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:16:49.745 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:16:49.745 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T09:16:51.043 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:16:51.879 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:51.879 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:16:51.894 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:51.894 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:16:51.894 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:16:51.894 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:16:51.894 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:16:51.894 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T09:16:51.895 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:51.895 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:16:51.895 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:16:51.895 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2019-12-09T09:16:51.895 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:16:51.895 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 50 M 2019-12-09T09:16:51.895 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 64 M 2019-12-09T09:16:51.896 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:16:51.897 INFO:teuthology.orchestra.run.smithi033.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:16:52.863 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:16:53.060 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:16:53.870 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:16:53.871 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:17:31.178 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:17:31.773 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:17:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:17:31.774 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:17:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T09:17:31.774 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:17:31.774 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:17:31.823 INFO:teuthology.orchestra.run.smithi033:> rpm -q kernel --last | head -n 1 2019-12-09T09:17:32.215 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:16:56 AM UTC 2019-12-09T09:17:32.217 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:17:32.217 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:17:32.218 INFO:teuthology.orchestra.run.smithi186:> uname -r 2019-12-09T09:17:32.341 INFO:teuthology.orchestra.run.smithi186.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:17:32.341 INFO:teuthology.task.kernel:Running kernel on smithi186: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:17:32.342 INFO:teuthology.orchestra.run.smithi186:> sudo yum install -y kernel 2019-12-09T09:17:32.592 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:17:32.633 INFO:teuthology.orchestra.run.smithi186.stdout:Determining fastest mirrors 2019-12-09T09:17:32.821 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:17:32.822 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T09:17:32.822 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirror.math.princeton.edu 2019-12-09T09:17:32.822 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:17:34.450 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:17:34.451 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:17:34.451 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T09:17:35.771 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:17:36.637 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:17:36.637 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:17:36.638 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:17:36.638 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:17:36.638 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2019-12-09T09:17:36.638 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:17:36.639 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:17:36.639 INFO:teuthology.orchestra.run.smithi186.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T09:17:36.639 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:17:36.639 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:17:36.639 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:17:36.639 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2019-12-09T09:17:36.639 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:17:36.640 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 50 M 2019-12-09T09:17:36.640 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 64 M 2019-12-09T09:17:36.640 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:17:36.640 INFO:teuthology.orchestra.run.smithi186.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:17:37.598 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:17:37.792 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:17:38.795 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:17:38.796 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:18:13.683 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:18:14.171 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T09:18:14.171 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:18:14.171 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:18:14.171 INFO:teuthology.orchestra.run.smithi186.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T09:18:14.171 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:18:14.171 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:18:14.232 INFO:teuthology.orchestra.run.smithi186:> rpm -q kernel --last | head -n 1 2019-12-09T09:18:14.561 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:17:41 AM UTC 2019-12-09T09:18:14.563 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:14.564 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:14.564 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-12-09T09:18:14.564 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T09:18:14.564 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T09:18:14.565 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-12-09T09:18:14.565 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T09:18:14.565 INFO:teuthology.orchestra.run.smithi033:> rpm -q kernel --last | head -n 1 2019-12-09T09:18:14.946 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:16:56 AM UTC 2019-12-09T09:18:14.948 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:14.948 INFO:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2 2019-12-09T09:18:15.090 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2 2019-12-09T09:18:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2019-12-09T09:18:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.02 2019-12-09T09:18:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 0.76.el7.centos 2019-12-09T09:18:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2019-12-09T09:18:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T09:18:15.091 INFO:teuthology.orchestra.run.smithi033.stdout:Group : System Environment/Base 2019-12-09T09:18:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 0 2019-12-09T09:18:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2019-12-09T09:18:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T09:18:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T09:18:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T09:18:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T09:18:15.092 INFO:teuthology.orchestra.run.smithi033.stdout:Relocations : (not relocatable) 2019-12-09T09:18:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : CentOS BuildSystem 2019-12-09T09:18:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2019-12-09T09:18:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T09:18:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T09:18:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2019-12-09T09:18:15.093 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:18:15.093 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T09:18:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T09:18:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T09:18:15.094 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2019-12-09T09:18:15.095 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T09:18:15.095 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:15.095 INFO:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T09:18:15.597 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2019-12-09T09:18:15.738 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:15.754 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T09:18:16.018 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:18:16.035 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T09:18:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T09:18:16.093 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T09:18:16.126 INFO:teuthology.orchestra.run.smithi033.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T09:18:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T09:18:17.129 INFO:teuthology.orchestra.run.smithi033.stderr:done 2019-12-09T09:18:17.131 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:18:17.197 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.HjajYBbXZC 2019-12-09T09:18:17.198 INFO:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HjajYBbXZC 2019-12-09T09:18:17.275 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.HjajYBbXZC 2019-12-09T09:18:17.439 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.HjajYBbXZC is 6KB 2019-12-09T09:18:17.539 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.HjajYBbXZC 2019-12-09T09:18:17.629 INFO:teuthology.orchestra.run.smithi033:> sudo grub2-set-default 0 2019-12-09T09:18:17.827 INFO:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2019-12-09T09:18:17.831 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-12-09T09:18:17.831 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi186.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T09:18:17.831 INFO:teuthology.orchestra.run.smithi186:> rpm -q kernel --last | head -n 1 2019-12-09T09:18:18.197 INFO:teuthology.orchestra.run.smithi186.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 09:17:41 AM UTC 2019-12-09T09:18:18.198 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:18.198 INFO:teuthology.orchestra.run.smithi186:> sudo rpm -qi grub2 2019-12-09T09:18:18.371 INFO:teuthology.orchestra.run.smithi186.stdout:Name : grub2 2019-12-09T09:18:18.372 INFO:teuthology.orchestra.run.smithi186.stdout:Epoch : 1 2019-12-09T09:18:18.372 INFO:teuthology.orchestra.run.smithi186.stdout:Version : 2.02 2019-12-09T09:18:18.372 INFO:teuthology.orchestra.run.smithi186.stdout:Release : 0.76.el7.centos 2019-12-09T09:18:18.372 INFO:teuthology.orchestra.run.smithi186.stdout:Architecture: x86_64 2019-12-09T09:18:18.373 INFO:teuthology.orchestra.run.smithi186.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T09:18:18.373 INFO:teuthology.orchestra.run.smithi186.stdout:Group : System Environment/Base 2019-12-09T09:18:18.373 INFO:teuthology.orchestra.run.smithi186.stdout:Size : 0 2019-12-09T09:18:18.373 INFO:teuthology.orchestra.run.smithi186.stdout:License : GPLv3+ 2019-12-09T09:18:18.374 INFO:teuthology.orchestra.run.smithi186.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T09:18:18.374 INFO:teuthology.orchestra.run.smithi186.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T09:18:18.374 INFO:teuthology.orchestra.run.smithi186.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T09:18:18.374 INFO:teuthology.orchestra.run.smithi186.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T09:18:18.375 INFO:teuthology.orchestra.run.smithi186.stdout:Relocations : (not relocatable) 2019-12-09T09:18:18.375 INFO:teuthology.orchestra.run.smithi186.stdout:Packager : CentOS BuildSystem 2019-12-09T09:18:18.375 INFO:teuthology.orchestra.run.smithi186.stdout:Vendor : CentOS 2019-12-09T09:18:18.375 INFO:teuthology.orchestra.run.smithi186.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T09:18:18.375 INFO:teuthology.orchestra.run.smithi186.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T09:18:18.376 INFO:teuthology.orchestra.run.smithi186.stdout:Description : 2019-12-09T09:18:18.376 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:18:18.376 INFO:teuthology.orchestra.run.smithi186.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T09:18:18.376 INFO:teuthology.orchestra.run.smithi186.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T09:18:18.377 INFO:teuthology.orchestra.run.smithi186.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T09:18:18.377 INFO:teuthology.orchestra.run.smithi186.stdout:hardware devices. 2019-12-09T09:18:18.378 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T09:18:18.378 INFO:teuthology.task.kernel:Updating grub on smithi186 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:18.379 INFO:teuthology.orchestra.run.smithi186:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T09:18:18.861 INFO:teuthology.orchestra.run.smithi186.stderr:Generating grub configuration file ... 2019-12-09T09:18:19.000 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:18:19.016 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T09:18:19.260 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T09:18:19.289 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T09:18:19.313 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T09:18:19.331 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T09:18:19.358 INFO:teuthology.orchestra.run.smithi186.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T09:18:19.367 INFO:teuthology.orchestra.run.smithi186.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T09:18:20.461 INFO:teuthology.orchestra.run.smithi186.stderr:done 2019-12-09T09:18:20.462 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:18:20.516 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.Wc0fe8im6Q 2019-12-09T09:18:20.516 INFO:teuthology.orchestra.run.smithi186:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Wc0fe8im6Q 2019-12-09T09:18:20.587 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.Wc0fe8im6Q 2019-12-09T09:18:20.764 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.Wc0fe8im6Q is 6KB 2019-12-09T09:18:20.834 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.Wc0fe8im6Q 2019-12-09T09:18:20.890 INFO:teuthology.orchestra.run.smithi186:> sudo grub2-set-default 0 2019-12-09T09:18:21.156 INFO:teuthology.orchestra.run.smithi186:> sudo shutdown -r now 2019-12-09T09:18:21.160 INFO:teuthology.misc:Re-opening connections... 2019-12-09T09:18:21.160 INFO:teuthology.misc:trying to connect to ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:18:21.160 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:18:21.161 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:18:21.162 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2019-12-09T09:18:21.162 INFO:teuthology.misc:trying to connect to ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:18:21.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:18:21.164 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:18:21.259 INFO:teuthology.orchestra.run.smithi186:> true 2019-12-09T09:18:21.588 DEBUG:teuthology.misc:waited 0.428364992142 2019-12-09T09:18:22.589 INFO:teuthology.misc:trying to connect to ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:18:22.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:18:22.590 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:19:22.591 DEBUG:teuthology.orchestra.remote:timed out 2019-12-09T09:19:22.591 DEBUG:teuthology.misc:waited 61.4309968948 2019-12-09T09:19:23.592 INFO:teuthology.misc:trying to connect to ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:19:23.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:19:23.593 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:19:55.064 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2019-12-09T09:19:55.065 DEBUG:teuthology.misc:waited 93.9047698975 2019-12-09T09:19:56.065 INFO:teuthology.misc:trying to connect to ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:19:56.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:19:56.066 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:19:56.067 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2019-12-09T09:19:56.067 DEBUG:teuthology.misc:waited 94.9073479176 2019-12-09T09:19:57.068 INFO:teuthology.misc:trying to connect to ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:19:57.068 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:19:57.069 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:19:57.182 INFO:teuthology.orchestra.run.smithi033:> true 2019-12-09T09:19:57.473 DEBUG:teuthology.misc:waited 96.3131008148 2019-12-09T09:19:58.474 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-12-09T09:19:58.475 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:19:58.475 INFO:teuthology.orchestra.run.smithi033:> uname -r 2019-12-09T09:19:58.561 INFO:teuthology.orchestra.run.smithi033.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:19:58.561 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:19:58.562 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T09:19:58.562 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-09T09:19:58.562 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:19:58.562 INFO:teuthology.orchestra.run.smithi186:> uname -r 2019-12-09T09:20:07.042 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-12-09T09:20:07.043 ERROR:teuthology.task.kernel:Saw exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/kernel.py", line 695, in wait_for_reboot assert not need_to_install(ctx, client, need_install[client]), \ File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/kernel.py", line 184, in need_to_install stdout=uname_fp, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 428, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 98, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 493, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/transport.py", line 806, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/transport.py", line 933, in open_channel raise e error: [Errno 104] Connection reset by peer 2019-12-09T09:20:08.045 INFO:teuthology.misc:Re-opening connections... 2019-12-09T09:20:08.045 INFO:teuthology.misc:trying to connect to ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:20:08.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:20:08.047 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:20:08.129 INFO:teuthology.orchestra.run.smithi033:> true 2019-12-09T09:20:08.449 DEBUG:teuthology.misc:waited 0.404096126556 2019-12-09T09:20:09.450 INFO:teuthology.misc:trying to connect to ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:20:09.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:20:09.451 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:20:09.537 INFO:teuthology.orchestra.run.smithi186:> true 2019-12-09T09:20:09.872 DEBUG:teuthology.misc:waited 1.82698011398 2019-12-09T09:20:10.873 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-09T09:20:10.874 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T09:20:10.874 INFO:teuthology.orchestra.run.smithi186:> uname -r 2019-12-09T09:20:10.959 INFO:teuthology.orchestra.run.smithi186.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:20:10.960 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T09:20:10.960 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T09:20:11.961 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-12-09T09:20:11.962 INFO:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T09:20:12.129 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2019-12-09T09:20:12.139 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-12-09T09:20:12.139 INFO:teuthology.orchestra.run.smithi186:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T09:20:12.204 INFO:teuthology.orchestra.run.smithi186.stdout:ttyS1 2019-12-09T09:20:12.217 INFO:teuthology.run_tasks:Running task internal.base... 2019-12-09T09:20:12.236 INFO:teuthology.task.internal:Creating test directory... 2019-12-09T09:20:12.237 INFO:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T09:20:12.241 INFO:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T09:20:12.331 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-12-09T09:20:12.339 INFO:teuthology.run_tasks:Running task internal.archive... 2019-12-09T09:20:12.351 INFO:teuthology.task.internal:Creating archive directory... 2019-12-09T09:20:12.354 INFO:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T09:20:12.361 INFO:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T09:20:12.444 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-12-09T09:20:12.453 INFO:teuthology.task.internal:Enabling coredump saving... 2019-12-09T09:20:12.454 INFO:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:20:12.458 INFO:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:20:12.510 INFO:teuthology.orchestra.run.smithi186.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:20:12.556 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T09:20:12.560 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-12-09T09:20:12.576 INFO:teuthology.task.internal:Configuring sudo... 2019-12-09T09:20:12.577 INFO:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T09:20:12.580 INFO:teuthology.orchestra.run.smithi186:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T09:20:12.676 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-12-09T09:20:12.699 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-12-09T09:20:12.699 INFO:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T09:20:12.704 INFO:teuthology.orchestra.run.smithi186:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T09:20:12.790 INFO:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:20:12.880 INFO:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:20:13.003 INFO:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:20:13.092 INFO:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:20:13.185 INFO:teuthology.orchestra.run.smithi033:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T09:20:13.274 INFO:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:20:13.331 INFO:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T09:20:13.394 INFO:teuthology.orchestra.run.smithi186:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:20:13.452 INFO:teuthology.orchestra.run.smithi186:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T09:20:13.509 INFO:teuthology.orchestra.run.smithi186:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T09:20:13.566 INFO:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2019-12-09T09:20:13.570 INFO:teuthology.orchestra.run.smithi186:> sudo service rsyslog restart 2019-12-09T09:20:13.678 INFO:teuthology.orchestra.run.smithi186.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T09:20:13.714 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T09:20:13.746 INFO:teuthology.run_tasks:Running task internal.timer... 2019-12-09T09:20:13.756 INFO:teuthology.task.internal:Starting timer... 2019-12-09T09:20:13.756 INFO:teuthology.run_tasks:Running task pcp... 2019-12-09T09:20:13.788 INFO:teuthology.run_tasks:Running task selinux... 2019-12-09T09:20:13.800 INFO:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2019-12-09T09:20:13.952 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: [ OK ] 2019-12-09T09:20:13.955 INFO:teuthology.orchestra.run.smithi186:> sudo service auditd rotate 2019-12-09T09:20:14.049 INFO:teuthology.orchestra.run.smithi186.stdout:Rotating logs: [ OK ] 2019-12-09T09:20:14.051 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-12-09T09:20:14.051 INFO:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2019-12-09T09:20:14.145 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2019-12-09T09:20:14.146 INFO:teuthology.orchestra.run.smithi186:> /usr/sbin/getenforce 2019-12-09T09:20:14.189 INFO:teuthology.orchestra.run.smithi186.stdout:Permissive 2019-12-09T09:20:14.190 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi186.front.sepia.ceph.com': 'permissive'} 2019-12-09T09:20:14.190 INFO:teuthology.orchestra.run.smithi033:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T09:20:14.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:20:14.311 INFO:teuthology.orchestra.run.smithi186:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T09:20:14.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:20:14.376 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-12-09T09:20:14.377 INFO:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2019-12-09T09:20:14.510 INFO:teuthology.orchestra.run.smithi186:> sudo /usr/sbin/setenforce permissive 2019-12-09T09:20:14.569 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-12-09T09:20:14.582 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2019-12-09T09:20:14.582 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-12-09T09:20:14.604 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-12-09T09:20:14.615 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi186.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-12-09T09:20:15.054 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-12-09T09:20:16.207 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2019-12-09T09:20:19.019 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-12-09T09:20:19.042 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:19 +0000 (0:00:00.065) 0:00:00.065 ******* 2019-12-09T09:20:19.055 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:19 +0000 (0:00:00.012) 0:00:00.077 ******* 2019-12-09T09:20:22.319 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi186.front.sepia.ceph.com] 2019-12-09T09:20:22.395 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:22 +0000 (0:00:03.340) 0:00:03.418 ******* 2019-12-09T09:20:22.418 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] 2019-12-09T09:20:22.499 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:22 +0000 (0:00:00.103) 0:00:03.521 ******* 2019-12-09T09:20:22.855 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-12-09T09:20:22.859 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:20:22.892 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:22 +0000 (0:00:00.393) 0:00:03.915 ******* 2019-12-09T09:20:22.898 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:20:22.935 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:22 +0000 (0:00:00.042) 0:00:03.957 ******* 2019-12-09T09:20:23.474 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-12-09T09:20:23.474 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T09:20:23.515 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:23 +0000 (0:00:00.579) 0:00:04.537 ******* 2019-12-09T09:20:23.522 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T09:20:23.559 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:23 +0000 (0:00:00.043) 0:00:04.581 ******* 2019-12-09T09:20:23.563 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-12-09T09:20:23.563 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:23.602 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:23 +0000 (0:00:00.043) 0:00:04.624 ******* 2019-12-09T09:20:23.607 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:23.647 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:23 +0000 (0:00:00.045) 0:00:04.669 ******* 2019-12-09T09:20:24.065 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2019-12-09T09:20:24.066 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:20:24.068 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:20:24.106 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:24 +0000 (0:00:00.459) 0:00:05.129 ******* 2019-12-09T09:20:24.146 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:24 +0000 (0:00:00.039) 0:00:05.169 ******* 2019-12-09T09:20:25.590 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2019-12-09T09:20:25.592 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883224.12-255811720840010/source", "state": "file", "uid": 0} 2019-12-09T09:20:25.631 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:25 +0000 (0:00:01.484) 0:00:06.654 ******* 2019-12-09T09:20:25.639 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883224.17-141075914965478/source", "state": "file", "uid": 0} 2019-12-09T09:20:25.677 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:25 +0000 (0:00:00.046) 0:00:06.700 ******* 2019-12-09T09:20:26.198 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi186.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:26.255 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:26.522 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:26.609 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:26.851 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:26.946 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:27.140 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:27.248 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:27.445 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:27.575 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:27.732 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:27.881 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:28.048 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:28.176 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T09:20:28.291 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** 2019-12-09T09:20:28.292 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2019-12-09T09:20:28.310 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-12-09T09:20:28.511 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:28 +0000 (0:00:02.833) 0:00:09.534 ******* 2019-12-09T09:20:28.555 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:28 +0000 (0:00:00.043) 0:00:09.578 ******* 2019-12-09T09:20:29.454 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi186.front.sepia.ceph.com] 2019-12-09T09:20:29.514 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] 2019-12-09T09:20:29.584 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-12-09T09:20:29.590 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:29 +0000 (0:00:01.028) 0:00:10.607 ******* 2019-12-09T09:20:29.634 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:29 +0000 (0:00:00.049) 0:00:10.656 ******* 2019-12-09T09:20:30.090 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-12-09T09:20:30.090 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.207719", "end": "2019-12-09 09:20:30.068226", "rc": 0, "start": "2019-12-09 09:20:29.860507", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T09:20:30.128 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.494) 0:00:11.151 ******* 2019-12-09T09:20:30.135 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.216807", "end": "2019-12-09 09:20:30.084503", "rc": 0, "start": "2019-12-09 09:20:29.867696", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T09:20:30.173 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.044) 0:00:11.196 ******* 2019-12-09T09:20:30.465 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-09T09:20:30.466 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.012318", "end": "2019-12-09 09:20:30.428544", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:20:30.416226", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T09:20:30.504 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.331) 0:00:11.527 ******* 2019-12-09T09:20:30.511 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013694", "end": "2019-12-09 09:20:30.466308", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:20:30.452614", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T09:20:30.548 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.044) 0:00:11.571 ******* 2019-12-09T09:20:30.554 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-09T09:20:30.554 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.593 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.044) 0:00:11.615 ******* 2019-12-09T09:20:30.600 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.639 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.046) 0:00:11.662 ******* 2019-12-09T09:20:30.645 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-09T09:20:30.647 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.686 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.046) 0:00:11.709 ******* 2019-12-09T09:20:30.696 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.738 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.051) 0:00:11.761 ******* 2019-12-09T09:20:30.743 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-09T09:20:30.744 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.782 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.043) 0:00:11.804 ******* 2019-12-09T09:20:30.787 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.825 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.043) 0:00:11.848 ******* 2019-12-09T09:20:30.831 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-09T09:20:30.833 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.869 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.043) 0:00:11.891 ******* 2019-12-09T09:20:30.875 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.913 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.044) 0:00:11.936 ******* 2019-12-09T09:20:30.919 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-09T09:20:30.920 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:30.976 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:30 +0000 (0:00:00.063) 0:00:11.999 ******* 2019-12-09T09:20:30.982 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:20:31.019 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:31 +0000 (0:00:00.042) 0:00:12.041 ******* 2019-12-09T09:20:31.314 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-12-09T09:20:31.315 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:20:31.353 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:31 +0000 (0:00:00.333) 0:00:12.375 ******* 2019-12-09T09:20:31.360 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:20:31.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:20:31 +0000 (0:00:00.045) 0:00:12.421 ******* 2019-12-09T09:20:31.770 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-09T09:20:31.771 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T09:20:31.812 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T09:20:32.119 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T09:20:32.153 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T09:20:32.451 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T09:20:32.522 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T09:20:32.795 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T09:20:32.859 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T09:20:33.150 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T09:20:33.192 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T09:20:33.449 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T09:20:33.471 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T09:20:33.734 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T09:20:33.803 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T09:20:34.073 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T09:20:34.126 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T09:20:34.411 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T09:20:34.444 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T09:20:34.741 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T09:20:34.798 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T09:20:35.025 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T09:20:35.157 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T09:20:35.365 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T09:20:35.480 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T09:20:35.698 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T09:20:35.839 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T09:20:36.064 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T09:20:36.182 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T09:20:36.403 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T09:20:36.520 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T09:20:36.729 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T09:20:36.865 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T09:20:37.070 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T09:20:37.218 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T09:20:37.401 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T09:20:37.537 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T09:20:37.748 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T09:20:37.863 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T09:20:38.106 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T09:20:38.184 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T09:20:38.452 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T09:20:38.485 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T09:20:38.783 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T09:20:38.807 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T09:20:39.098 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T09:20:39.132 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T09:20:39.400 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T09:20:39.419 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T09:20:39.746 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T09:20:39.778 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T09:20:40.092 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T09:20:40.114 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T09:20:40.427 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T09:20:40.428 INFO:teuthology.task.ansible.out: 2019-12-09T09:20:40.439 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T09:20:40.751 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T09:20:40.773 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T09:20:41.085 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T09:20:41.097 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T09:20:41.413 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T09:20:41.447 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T09:20:41.744 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T09:20:41.778 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T09:20:42.092 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T09:20:42.143 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T09:20:42.407 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T09:20:42.472 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T09:20:42.756 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T09:20:42.819 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T09:20:43.032 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T09:20:43.151 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T09:20:43.315 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T09:20:43.463 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T09:20:43.647 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T09:20:43.796 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T09:20:43.965 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T09:20:44.143 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T09:20:44.256 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T09:20:44.475 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T09:20:44.591 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:20:44.820 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:20:44.935 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T09:20:45.154 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T09:20:45.269 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T09:20:45.499 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T09:20:45.634 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T09:20:45.843 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T09:20:45.962 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T09:20:46.174 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T09:20:46.322 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T09:20:46.472 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T09:20:46.655 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T09:20:46.815 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T09:20:46.982 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T09:20:47.156 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T09:20:47.272 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:20:47.498 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:20:47.613 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T09:20:47.833 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T09:20:47.958 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T09:20:48.178 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T09:20:48.286 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T09:20:48.538 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T09:20:48.623 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T09:20:48.882 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T09:20:48.938 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T09:20:49.234 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T09:20:49.266 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T09:20:49.267 INFO:teuthology.task.ansible.out: 2019-12-09T09:20:49.587 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T09:20:49.588 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T09:20:49.886 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T09:20:49.939 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T09:20:50.207 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T09:20:50.291 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T09:20:50.534 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T09:20:50.659 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T09:20:50.883 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T09:20:51.009 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T09:20:51.229 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T09:20:51.335 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T09:20:51.587 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T09:20:51.588 INFO:teuthology.task.ansible.out: 2019-12-09T09:20:51.650 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T09:20:51.932 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T09:20:52.005 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T09:20:52.275 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T09:20:52.332 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T09:20:52.625 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T09:20:52.679 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T09:20:52.949 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:20:53.033 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T09:20:53.248 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T09:20:53.363 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T09:20:53.583 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T09:20:53.712 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T09:20:53.910 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T09:20:54.069 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T09:20:54.250 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T09:20:54.398 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T09:20:54.578 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T09:20:54.746 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T09:20:54.913 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T09:20:55.091 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T09:20:55.266 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T09:20:55.433 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T09:20:55.572 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T09:20:55.777 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T09:20:55.896 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T09:20:56.131 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T09:20:56.227 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T09:20:56.449 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T09:20:56.555 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T09:20:56.809 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T09:20:56.888 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T09:20:57.147 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T09:20:57.224 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T09:20:57.522 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T09:20:57.555 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T09:20:57.879 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T09:20:57.891 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T09:20:58.235 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T09:20:58.237 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T09:20:58.567 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T09:20:58.599 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T09:20:58.922 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T09:20:58.955 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T09:20:59.276 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T09:20:59.277 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T09:20:59.586 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T09:20:59.632 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T09:20:59.907 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T09:20:59.971 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T09:21:00.254 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T09:21:00.327 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T09:21:00.603 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T09:21:00.678 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T09:21:00.939 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T09:21:01.024 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T09:21:01.266 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T09:21:01.371 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T09:21:01.550 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T09:21:01.751 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T09:21:01.869 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T09:21:02.087 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T09:21:02.192 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T09:21:02.432 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T09:21:02.538 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T09:21:02.770 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T09:21:02.834 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T09:21:03.100 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T09:21:03.165 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T09:21:03.439 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T09:21:03.508 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T09:21:03.780 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T09:21:03.870 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T09:21:04.134 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T09:21:04.236 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T09:21:04.477 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T09:21:04.552 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T09:21:04.831 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T09:21:04.875 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T09:21:05.166 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T09:21:05.190 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T09:21:05.502 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T09:21:05.536 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:21:05.839 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T09:21:05.870 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T09:21:06.168 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T09:21:06.509 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T09:21:06.612 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T09:21:06.761 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:06 +0000 (0:00:35.361) 0:00:47.783 ******* 2019-12-09T09:21:06.787 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-09T09:21:06.850 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:06 +0000 (0:00:00.089) 0:00:47.872 ******* 2019-12-09T09:21:06.856 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-09T09:21:06.857 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T09:21:06.904 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T09:21:06.915 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-09T09:21:06.916 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-09T09:21:06.918 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-09T09:21:06.919 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-09T09:21:06.921 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-09T09:21:06.922 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-09T09:21:06.924 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 2019-12-09T09:21:06.925 INFO:teuthology.task.ansible.out:6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f8 2019-12-09T09:21:06.927 INFO:teuthology.task.ansible.out:48846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8a 2019-12-09T09:21:06.928 INFO:teuthology.task.ansible.out:a01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalho 2019-12-09T09:21:06.930 INFO:teuthology.task.ansible.out:jr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T09:21:06.972 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:06 +0000 (0:00:00.122) 0:00:47.995 ******* 2019-12-09T09:21:07.482 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-09T09:21:07.483 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T09:21:07.678 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:07 +0000 (0:00:00.705) 0:00:48.700 ******* 2019-12-09T09:21:07.706 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-09T09:21:07.747 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:07 +0000 (0:00:00.069) 0:00:48.770 ******* 2019-12-09T09:21:07.797 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T09:21:07.802 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-09T09:21:07.804 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-09T09:21:07.806 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-09T09:21:07.807 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-09T09:21:07.808 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-09T09:21:07.859 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-09T09:21:07.861 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 2019-12-09T09:21:07.864 INFO:teuthology.task.ansible.out:6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f8 2019-12-09T09:21:07.865 INFO:teuthology.task.ansible.out:48846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8a 2019-12-09T09:21:07.867 INFO:teuthology.task.ansible.out:a01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1P 2019-12-09T09:21:07.868 INFO:teuthology.task.ansible.out:nHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4 2019-12-09T09:21:07.870 INFO:teuthology.task.ansible.out:aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T09:21:07.937 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:07 +0000 (0:00:00.190) 0:00:48.960 ******* 2019-12-09T09:21:07.986 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-09T09:21:07.988 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T09:21:08.036 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:08 +0000 (0:00:00.098) 0:00:49.059 ******* 2019-12-09T09:21:08.446 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-09T09:21:08.662 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-09T09:21:08.664 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T09:21:08.707 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:08 +0000 (0:00:00.670) 0:00:49.729 ******* 2019-12-09T09:21:08.828 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:21:08.829 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T09:21:09.113 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-09T09:21:09.198 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T09:21:09.487 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T09:21:09.582 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T09:21:09.863 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T09:21:09.949 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-09T09:21:09.950 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-09T09:21:10.202 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T09:21:10.299 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-09T09:21:10.541 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-09T09:21:10.545 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-09T09:21:10.648 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T09:21:10.650 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T09:21:10.899 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-09T09:21:10.952 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T09:21:11.254 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T09:21:11.255 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T09:21:11.322 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T09:21:11.333 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:11.613 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T09:21:11.687 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T09:21:11.688 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T09:21:11.698 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:11.711 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:11.739 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:11.977 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T09:21:11.988 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:12.056 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T09:21:12.057 INFO:teuthology.task.ansible.out: 2019-12-09T09:21:12.352 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T09:21:12.353 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T09:21:12.373 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:12.374 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:12.376 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-09T09:21:12.380 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T09:21:12.387 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:12.710 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T09:21:12.770 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T09:21:13.084 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-09T09:21:13.086 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T09:21:13.097 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T09:21:13.392 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T09:21:13.440 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T09:21:13.721 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T09:21:13.755 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T09:21:14.040 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T09:21:14.128 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T09:21:14.387 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T09:21:14.498 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T09:21:14.746 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T09:21:14.747 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T09:21:14.825 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T09:21:15.100 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T09:21:15.102 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T09:21:15.175 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T09:21:15.469 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T09:21:15.553 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T09:21:15.555 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T09:21:15.782 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T09:21:15.886 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T09:21:15.888 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T09:21:16.151 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T09:21:16.152 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T09:21:16.224 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T09:21:16.502 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T09:21:16.568 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T09:21:16.840 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T09:21:16.925 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T09:21:16.926 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T09:21:17.166 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T09:21:17.301 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T09:21:17.524 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T09:21:17.640 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T09:21:17.857 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T09:21:17.995 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T09:21:18.218 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T09:21:18.220 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T09:21:18.337 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T09:21:18.542 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T09:21:18.688 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T09:21:18.889 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T09:21:19.055 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T09:21:19.057 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T09:21:19.259 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T09:21:19.407 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T09:21:19.607 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T09:21:19.754 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T09:21:19.923 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-09T09:21:20.097 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T09:21:20.247 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T09:21:20.458 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T09:21:20.596 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:21:20.817 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": tr 2019-12-09T09:21:20.818 INFO:teuthology.task.ansible.out:ue} 2019-12-09T09:21:20.819 INFO:teuthology.task.ansible.out: 2019-12-09T09:21:20.931 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T09:21:21.157 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T09:21:21.302 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T09:21:21.519 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:21:21.617 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T09:21:21.618 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-09T09:21:21.866 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T09:21:21.945 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T09:21:21.947 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T09:21:22.236 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T09:21:22.280 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-09T09:21:22.616 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T09:21:22.619 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T09:21:22.973 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T09:21:23.006 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T09:21:23.007 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T09:21:23.349 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1Jf 2019-12-09T09:21:23.352 INFO:teuthology.task.ansible.out:sI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:21:23.689 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T09:21:23.690 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T09:21:23.692 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-09T09:21:23.713 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T09:21:24.054 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T09:21:24.085 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T09:21:24.385 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T09:21:24.428 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-09T09:21:24.430 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:21:24.703 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T09:21:24.803 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T09:21:24.804 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T09:21:24.806 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-09T09:21:25.059 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T09:21:25.166 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T09:21:25.402 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T09:21:25.521 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T09:21:25.757 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T09:21:25.885 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T09:21:26.129 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T09:21:26.234 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T09:21:26.456 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T09:21:26.603 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T09:21:26.791 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-09T09:21:26.793 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-09T09:21:26.952 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T09:21:27.137 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T09:21:27.307 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T09:21:27.488 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T09:21:27.679 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T09:21:27.841 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-09T09:21:27.843 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T09:21:28.066 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-09T09:21:28.067 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-09T09:21:28.171 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T09:21:28.415 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T09:21:28.511 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T09:21:28.792 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T09:21:28.871 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T09:21:29.195 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-09T09:21:29.197 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T09:21:29.208 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:21:29.564 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL 2019-12-09T09:21:29.566 INFO:teuthology.task.ansible.out:+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T09:21:29.906 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T09:21:29.927 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T09:21:30.259 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T09:21:30.299 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T09:21:30.625 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T09:21:30.658 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T09:21:30.915 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T09:21:31.010 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-09T09:21:31.199 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T09:21:31.361 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T09:21:31.465 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T09:21:31.709 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T09:21:31.773 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T09:21:32.048 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T09:21:32.102 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T09:21:32.411 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T09:21:32.475 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T09:21:32.746 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T09:21:32.850 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T09:21:33.101 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T09:21:33.225 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T09:21:33.443 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T09:21:33.592 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T09:21:33.806 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T09:21:33.945 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T09:21:34.192 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T09:21:34.297 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T09:21:34.538 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T09:21:34.653 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T09:21:34.861 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T09:21:34.992 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T09:21:35.181 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T09:21:35.358 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T09:21:35.533 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T09:21:35.719 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T09:21:35.721 INFO:teuthology.task.ansible.out: 2019-12-09T09:21:35.896 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T09:21:36.080 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T09:21:36.241 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T09:21:36.403 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T09:21:36.593 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOF 2019-12-09T09:21:36.594 INFO:teuthology.task.ansible.out:LIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T09:21:36.773 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T09:21:36.775 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-09T09:21:36.941 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T09:21:37.131 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T09:21:37.132 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T09:21:37.289 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T09:21:37.508 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T09:21:37.665 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T09:21:37.866 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T09:21:38.032 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T09:21:38.226 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T09:21:38.383 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T09:21:38.386 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-09T09:21:38.591 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T09:21:38.734 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T09:21:38.736 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T09:21:38.975 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T09:21:39.088 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T09:21:39.328 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T09:21:39.442 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T09:21:39.686 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T09:21:39.781 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T09:21:40.040 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T09:21:40.135 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T09:21:40.380 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T09:21:40.508 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T09:21:40.733 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T09:21:40.860 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T09:21:41.105 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T09:21:41.229 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T09:21:41.458 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-09T09:21:41.572 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T09:21:41.793 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T09:21:41.927 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T09:21:42.160 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T09:21:42.305 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T09:21:42.494 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-09T09:21:42.495 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:21:42.669 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T09:21:42.848 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T09:21:43.014 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-09T09:21:43.229 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-09T09:21:43.351 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:43 +0000 (0:00:34.642) 0:01:24.372 ******* 2019-12-09T09:21:43.394 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T09:21:43.404 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:43.406 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.416 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.428 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.439 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.440 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.451 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.462 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.463 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.479 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:43.713 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T09:21:43.851 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T09:21:43.862 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.073 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsri 2019-12-09T09:21:44.075 INFO:teuthology.task.ansible.out:vas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T09:21:44.203 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:44.205 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T09:21:44.447 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T09:21:44.553 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:44.553 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T09:21:44.795 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T09:21:44.797 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-09T09:21:44.852 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:44.854 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T09:21:44.892 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:44 +0000 (0:00:01.541) 0:01:25.914 ******* 2019-12-09T09:21:44.899 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.901 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.908 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.919 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.930 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.942 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.944 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.955 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.956 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.958 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.968 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.970 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.980 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.992 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:44.993 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.004 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.005 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.006 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.017 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.028 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.040 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.051 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.062 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.064 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.074 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.085 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.097 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.098 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.109 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.120 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.131 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.142 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.153 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.168 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.196 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.197 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.198 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.209 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.229 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.231 INFO:teuthology.task.ansible.out: skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.242 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.253 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.264 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.276 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.290 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.298 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.309 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.320 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.332 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.343 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.354 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.355 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.366 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.378 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.389 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T09:21:45.391 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.402 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.403 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.404 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.414 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.415 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.426 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.428 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.439 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.450 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.451 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.461 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.473 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.473 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.484 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.495 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.517 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.517 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.534 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.540 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.551 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.563 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.563 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.574 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.585 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.586 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.597 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.608 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.619 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.630 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.630 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.641 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.658 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.664 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.675 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.686 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.698 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.719 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T09:21:45.757 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:45 +0000 (0:00:00.864) 0:01:26.779 ******* 2019-12-09T09:21:45.799 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-09T09:21:45.800 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:45.837 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:45 +0000 (0:00:00.080) 0:01:26.860 ******* 2019-12-09T09:21:46.056 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:46.059 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T09:21:46.214 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:21:46.216 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-09T09:21:46.357 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:46.358 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T09:21:46.368 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.383 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.394 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.416 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.427 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.428 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.440 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.451 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.462 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.473 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.484 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.484 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.502 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.508 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.518 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.519 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.530 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.541 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.548 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi186.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-09T09:21:46.553 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:46.554 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.569 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.570 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.581 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.582 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.593 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.604 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.605 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.630 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.635 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.636 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.647 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.648 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.659 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.670 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.681 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.682 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.692 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.703 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.714 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.725 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.748 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.749 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.759 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.770 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.771 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.793 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.795 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.807 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.819 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.831 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.832 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.843 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.863 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi186.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.881 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.882 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.894 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.906 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.911 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.919 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.930 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.942 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.954 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.965 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.966 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.979 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:46.989 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.008 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.013 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.024 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.036 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.049 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.069 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.070 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.086 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.092 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.103 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.125 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.126 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.137 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.138 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.149 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.164 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.174 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.182 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.183 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.195 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.208 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.223 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi186.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T09:21:47.230 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T09:21:47.231 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.245 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.257 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.310 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:47 +0000 (0:00:01.472) 0:01:28.333 ******* 2019-12-09T09:21:47.348 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-09T09:21:47.349 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:21:47.393 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:21:47 +0000 (0:00:00.082) 0:01:28.416 ******* 2019-12-09T09:21:47.579 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:21:47.579 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-09T09:21:47.757 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-09T09:21:47.939 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T09:21:48.098 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-09T09:21:48.435 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-09T09:21:48.499 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T09:21:48.776 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T09:21:48.812 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T09:21:49.106 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-09T09:21:49.129 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T09:21:49.439 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T09:21:49.461 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T09:21:49.783 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T09:21:49.795 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T09:21:50.116 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T09:21:50.127 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T09:21:50.442 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T09:21:50.454 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T09:21:50.804 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T09:21:50.816 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T09:21:51.113 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T09:21:51.167 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T09:21:51.420 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T09:21:51.475 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T09:21:51.755 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T09:21:51.808 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T09:21:52.085 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T09:21:52.143 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T09:21:52.414 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T09:21:52.500 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T09:21:52.760 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T09:21:52.844 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T09:21:53.059 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T09:21:53.202 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T09:21:53.526 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T09:21:53.606 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T09:21:53.863 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T09:21:54.206 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T09:21:54.217 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T09:21:54.568 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T09:21:54.771 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T09:21:55.189 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T09:21:55.395 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T09:21:55.776 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T09:21:55.937 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T09:21:56.399 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T09:21:56.447 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T09:21:57.013 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T09:21:57.067 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T09:21:57.631 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T09:21:57.643 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T09:21:58.162 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T09:21:58.242 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T09:21:58.749 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T09:21:58.792 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T09:21:59.360 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T09:21:59.404 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T09:21:59.930 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T09:22:00.028 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T09:22:00.359 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T09:22:00.441 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T09:22:00.712 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T09:22:01.028 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T09:22:01.061 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T09:22:01.374 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T09:22:01.679 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T09:22:01.703 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T09:22:02.027 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T09:22:02.029 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T09:22:02.360 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T09:22:02.364 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T09:22:02.696 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T09:22:02.993 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T09:22:03.046 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T09:22:03.145 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:03 +0000 (0:00:15.751) 0:01:44.167 ******* 2019-12-09T09:22:03.338 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T09:22:03.654 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T09:22:03.967 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T09:22:04.301 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T09:22:04.303 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883323.17-61540139302073/source", "state": "file", "uid": 0} 2019-12-09T09:22:04.338 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:04 +0000 (0:00:01.193) 0:01:45.361 ******* 2019-12-09T09:22:04.376 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-09T09:22:04.378 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => { "msg": "Host smithi186.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T09:22:04.416 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:04 +0000 (0:00:00.077) 0:01:45.439 ******* 2019-12-09T09:22:04.453 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-09T09:22:04.455 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T09:22:04.494 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:04 +0000 (0:00:00.078) 0:01:45.517 ******* 2019-12-09T09:22:04.525 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T09:22:04.527 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T09:22:04.548 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:04.586 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:04 +0000 (0:00:00.091) 0:01:45.608 ******* 2019-12-09T09:22:04.618 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:04 +0000 (0:00:00.032) 0:01:45.641 ******* 2019-12-09T09:22:05.051 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.103564", "end": "2019-12-09 09:22:05.017238", "rc": 0, "start": "2019-12-09 09:22:04.913674", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T09:22:05.081 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.462) 0:01:46.103 ******* 2019-12-09T09:22:05.106 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-09T09:22:05.107 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.147 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.066) 0:01:46.170 ******* 2019-12-09T09:22:05.173 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-09T09:22:05.174 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.208 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.061) 0:01:46.231 ******* 2019-12-09T09:22:05.234 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-09T09:22:05.236 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.270 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.061) 0:01:46.293 ******* 2019-12-09T09:22:05.297 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-09T09:22:05.298 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.337 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.066) 0:01:46.360 ******* 2019-12-09T09:22:05.376 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-09T09:22:05.376 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.412 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.075) 0:01:46.435 ******* 2019-12-09T09:22:05.448 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-09T09:22:05.450 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.489 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.077) 0:01:46.512 ******* 2019-12-09T09:22:05.525 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-09T09:22:05.526 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:05.527 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:05.577 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.087) 0:01:46.599 ******* 2019-12-09T09:22:05.605 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.645 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.064) 0:01:46.664 ******* 2019-12-09T09:22:05.667 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-09T09:22:05.669 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.711 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.069) 0:01:46.733 ******* 2019-12-09T09:22:05.737 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-09T09:22:05.738 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:05.776 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.064) 0:01:46.798 ******* 2019-12-09T09:22:05.781 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T09:22:05.781 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883324.61-27797336334075/source", "state": "file", "uid": 0} 2019-12-09T09:22:05.826 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.045) 0:01:46.843 ******* 2019-12-09T09:22:05.828 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:05.876 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.056) 0:01:46.899 ******* 2019-12-09T09:22:05.882 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-09T09:22:05.884 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => { "msg": "Host smithi033.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T09:22:05.920 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.043) 0:01:46.943 ******* 2019-12-09T09:22:05.926 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-09T09:22:05.928 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:05.966 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:05 +0000 (0:00:00.045) 0:01:46.988 ******* 2019-12-09T09:22:05.971 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-09T09:22:05.972 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T09:22:06.009 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.043) 0:01:47.032 ******* 2019-12-09T09:22:06.015 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T09:22:06.015 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.053 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.043) 0:01:47.075 ******* 2019-12-09T09:22:06.058 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-09T09:22:06.064 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.101 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.048) 0:01:47.123 ******* 2019-12-09T09:22:06.106 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-09T09:22:06.111 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.149 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.048) 0:01:47.172 ******* 2019-12-09T09:22:06.187 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-09T09:22:06.188 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.229 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.080) 0:01:47.252 ******* 2019-12-09T09:22:06.256 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T09:22:06.257 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.304 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.074) 0:01:47.326 ******* 2019-12-09T09:22:06.338 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-09T09:22:06.339 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:06.382 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.078) 0:01:47.405 ******* 2019-12-09T09:22:06.420 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T09:22:06.422 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.459 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.076) 0:01:47.482 ******* 2019-12-09T09:22:06.501 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.544 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.084) 0:01:47.567 ******* 2019-12-09T09:22:06.550 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-09T09:22:06.551 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.097357", "end": "2019-12-09 09:22:06.489656", "rc": 0, "start": "2019-12-09 09:22:06.392299", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T09:22:06.590 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.045) 0:01:47.612 ******* 2019-12-09T09:22:06.596 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-09T09:22:06.598 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.635 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.044) 0:01:47.657 ******* 2019-12-09T09:22:06.641 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-09T09:22:06.643 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.680 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.044) 0:01:47.702 ******* 2019-12-09T09:22:06.687 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.726 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.044) 0:01:47.747 ******* 2019-12-09T09:22:06.730 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-09T09:22:06.731 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.776 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.051) 0:01:47.798 ******* 2019-12-09T09:22:06.783 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-09T09:22:06.783 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.823 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.047) 0:01:47.846 ******* 2019-12-09T09:22:06.830 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-09T09:22:06.830 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.870 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.046) 0:01:47.893 ******* 2019-12-09T09:22:06.888 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.917 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.047) 0:01:47.940 ******* 2019-12-09T09:22:06.923 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-09T09:22:06.923 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:06.967 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:06 +0000 (0:00:00.045) 0:01:47.986 ******* 2019-12-09T09:22:06.977 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi186.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:07.008 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.044) 0:01:48.030 ******* 2019-12-09T09:22:07.015 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-09T09:22:07.015 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.061 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.053) 0:01:48.083 ******* 2019-12-09T09:22:07.072 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T09:22:07.108 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.046) 0:01:48.130 ******* 2019-12-09T09:22:07.113 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-09T09:22:07.115 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.154 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.045) 0:01:48.176 ******* 2019-12-09T09:22:07.159 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-09T09:22:07.160 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.202 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.047) 0:01:48.224 ******* 2019-12-09T09:22:07.217 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi033.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi033.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:07.249 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.047) 0:01:48.272 ******* 2019-12-09T09:22:07.275 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-09T09:22:07.277 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.312 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.062) 0:01:48.335 ******* 2019-12-09T09:22:07.339 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-09T09:22:07.341 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.378 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.065) 0:01:48.401 ******* 2019-12-09T09:22:07.421 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi033.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:07.452 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.074) 0:01:48.475 ******* 2019-12-09T09:22:07.480 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-09T09:22:07.482 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.517 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.065) 0:01:48.540 ******* 2019-12-09T09:22:07.546 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-09T09:22:07.548 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:07.585 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.067) 0:01:48.608 ******* 2019-12-09T09:22:07.595 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-09T09:22:07.596 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:22:07.641 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.056) 0:01:48.664 ******* 2019-12-09T09:22:07.647 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T09:22:07.647 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.686 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.044) 0:01:48.709 ******* 2019-12-09T09:22:07.724 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-09T09:22:07.725 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.769 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.083) 0:01:48.792 ******* 2019-12-09T09:22:07.807 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-09T09:22:07.808 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.847 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.077) 0:01:48.870 ******* 2019-12-09T09:22:07.884 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T09:22:07.886 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:07.924 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.075) 0:01:48.945 ******* 2019-12-09T09:22:07.949 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-09T09:22:07.951 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:07.985 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:07 +0000 (0:00:00.062) 0:01:49.008 ******* 2019-12-09T09:22:08.012 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T09:22:08.014 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:08.055 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.069) 0:01:49.078 ******* 2019-12-09T09:22:08.082 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-09T09:22:08.084 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:08.126 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.070) 0:01:49.148 ******* 2019-12-09T09:22:08.163 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:08.208 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.082) 0:01:49.231 ******* 2019-12-09T09:22:08.245 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-12-09T09:22:08.274 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:08.283 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.074) 0:01:49.305 ******* 2019-12-09T09:22:08.321 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-09T09:22:08.323 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:08.358 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.075) 0:01:49.381 ******* 2019-12-09T09:22:08.388 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-09T09:22:08.390 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:08.445 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.086) 0:01:49.467 ******* 2019-12-09T09:22:08.472 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-09T09:22:08.474 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T09:22:08.525 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.080) 0:01:49.547 ******* 2019-12-09T09:22:08.553 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T09:22:08.591 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.066) 0:01:49.614 ******* 2019-12-09T09:22:08.619 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-09T09:22:08.621 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:08.666 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:08 +0000 (0:00:00.074) 0:01:49.689 ******* 2019-12-09T09:22:08.888 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-09T09:22:08.889 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T09:22:09.046 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-09T09:22:09.047 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:22:09.092 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:09 +0000 (0:00:00.425) 0:01:50.115 ******* 2019-12-09T09:22:09.824 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi186.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T09:22:09.875 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:09 +0000 (0:00:00.782) 0:01:50.897 ******* 2019-12-09T09:22:10.031 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T09:22:11.017 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T09:22:11.085 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:11 +0000 (0:00:01.210) 0:01:52.108 ******* 2019-12-09T09:22:11.092 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T09:22:11.093 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883329.92-163038747437439/source", "state": "file", "uid": 0} 2019-12-09T09:22:12.137 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883331.06-116552401377200/source", "state": "file", "uid": 0} 2019-12-09T09:22:12.178 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:12 +0000 (0:00:01.093) 0:01:53.201 ******* 2019-12-09T09:22:12.284 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883331.13-239694699439296/source", "state": "file", "uid": 0} 2019-12-09T09:22:12.781 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-09T09:22:12.782 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T09:22:13.032 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:22:13.036 INFO:teuthology.task.ansible.out: [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-12-09T09:22:13.075 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:13 +0000 (0:00:00.896) 0:01:54.098 ******* 2019-12-09T09:22:13.371 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T09:22:13.372 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575883332.3-2621577616550/source", "state": "file", "uid": 0} 2019-12-09T09:22:13.410 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:13 +0000 (0:00:00.335) 0:01:54.433 ******* 2019-12-09T09:22:13.754 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-09T09:22:13.755 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T09:22:14.066 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:22:14.105 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:14 +0000 (0:00:00.694) 0:01:55.127 ******* 2019-12-09T09:22:26.052 INFO:teuthology.task.ansible.out: 2019-12-09T09:22:26.054 INFO:teuthology.task.ansible.out:TASK [common : Update yum cache] *********************************************** [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.670662", "end": "2019-12-09 09:22:26.019959", "rc": 0, "start": "2019-12-09 09:22:13.349297", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.ash.fastserv.com\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.ash.fastserv.com", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T09:22:26.098 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:26 +0000 (0:00:11.992) 0:02:07.120 ******* 2019-12-09T09:22:26.099 INFO:teuthology.task.ansible.out: 2019-12-09T09:22:26.446 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T09:22:26.447 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T09:22:26.763 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:22:26.805 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:26 +0000 (0:00:00.707) 0:02:07.828 ******* 2019-12-09T09:22:26.832 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:22:26.834 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:26.870 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:26 +0000 (0:00:00.064) 0:02:07.893 ******* 2019-12-09T09:22:26.887 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-12-09T09:22:26.888 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.444099", "end": "2019-12-09 09:22:26.852124", "rc": 0, "start": "2019-12-09 09:22:14.408025", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.fileplanet.com\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.fileplanet.com", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T09:22:26.933 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:26 +0000 (0:00:00.062) 0:02:07.956 ******* 2019-12-09T09:22:26.940 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-09T09:22:26.941 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:26.980 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:26 +0000 (0:00:00.047) 0:02:08.003 ******* 2019-12-09T09:22:27.017 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T09:22:27.018 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.069 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.088) 0:02:08.091 ******* 2019-12-09T09:22:27.105 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:22:27.107 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.144 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.075) 0:02:08.167 ******* 2019-12-09T09:22:27.179 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-09T09:22:27.180 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.221 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.075) 0:02:08.243 ******* 2019-12-09T09:22:27.246 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-09T09:22:27.248 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.285 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.064) 0:02:08.307 ******* 2019-12-09T09:22:27.308 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T09:22:27.309 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T09:22:27.311 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.349 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.063) 0:02:08.372 ******* 2019-12-09T09:22:27.376 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.414 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.064) 0:02:08.436 ******* 2019-12-09T09:22:27.621 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T09:22:27.663 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.249) 0:02:08.686 ******* 2019-12-09T09:22:27.690 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:22:27.692 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.731 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.067) 0:02:08.753 ******* 2019-12-09T09:22:27.757 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-09T09:22:27.762 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.800 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.068) 0:02:08.822 ******* 2019-12-09T09:22:27.837 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T09:22:27.839 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.876 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.076) 0:02:08.899 ******* 2019-12-09T09:22:27.903 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T09:22:27.904 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:27.943 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:27 +0000 (0:00:00.065) 0:02:08.964 ******* 2019-12-09T09:22:27.969 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:28.011 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:28 +0000 (0:00:00.069) 0:02:09.034 ******* 2019-12-09T09:22:28.038 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-09T09:22:28.039 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:28.075 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:28 +0000 (0:00:00.063) 0:02:09.097 ******* 2019-12-09T09:22:28.102 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-09T09:22:28.103 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:28.138 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:28 +0000 (0:00:00.063) 0:02:09.160 ******* 2019-12-09T09:22:28.165 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-09T09:22:28.166 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:28.202 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:28 +0000 (0:00:00.064) 0:02:09.225 ******* 2019-12-09T09:22:28.528 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-09T09:22:28.529 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.805319", "end": "2019-12-09 09:22:28.500492", "rc": 0, "start": "2019-12-09 09:22:27.695173", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:22:28.567 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:28 +0000 (0:00:00.364) 0:02:09.590 ******* 2019-12-09T09:22:29.386 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-09T09:22:29.388 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T09:22:29.444 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:29 +0000 (0:00:00.876) 0:02:10.467 ******* 2019-12-09T09:22:29.471 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T09:22:29.473 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:29.512 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:29 +0000 (0:00:00.068) 0:02:10.535 ******* 2019-12-09T09:22:29.528 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-09T09:22:29.529 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.995208", "end": "2019-12-09 09:22:29.500466", "rc": 0, "start": "2019-12-09 09:22:28.505258", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:22:29.569 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:29 +0000 (0:00:00.057) 0:02:10.592 ******* 2019-12-09T09:22:29.578 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T09:22:29.579 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:29.638 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:29 +0000 (0:00:00.067) 0:02:10.660 ******* 2019-12-09T09:22:29.664 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T09:22:29.666 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:29.709 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:29 +0000 (0:00:00.071) 0:02:10.732 ******* 2019-12-09T09:22:29.739 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T09:22:29.741 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:29.775 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:29 +0000 (0:00:00.065) 0:02:10.798 ******* 2019-12-09T09:22:30.097 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-09T09:22:30.099 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T09:22:30.136 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:30 +0000 (0:00:00.360) 0:02:11.158 ******* 2019-12-09T09:22:30.165 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T09:22:30.166 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:30.205 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:30 +0000 (0:00:00.069) 0:02:11.228 ******* 2019-12-09T09:22:30.243 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T09:22:30.244 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:30.291 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:30 +0000 (0:00:00.086) 0:02:11.314 ******* 2019-12-09T09:22:30.326 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T09:22:30.328 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:30.362 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:30 +0000 (0:00:00.070) 0:02:11.384 ******* 2019-12-09T09:22:30.516 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T09:22:30.517 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:30.559 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:30 +0000 (0:00:00.197) 0:02:11.582 ******* 2019-12-09T09:22:30.767 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-09T09:22:30.769 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T09:22:30.821 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:30 +0000 (0:00:00.261) 0:02:11.844 ******* 2019-12-09T09:22:31.645 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T09:22:31.694 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:31 +0000 (0:00:00.873) 0:02:12.717 ******* 2019-12-09T09:22:34.842 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-09T09:22:34.844 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T09:22:34.881 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:34 +0000 (0:00:03.186) 0:02:15.903 ******* 2019-12-09T09:22:35.778 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T09:22:35.778 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T09:22:35.812 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-09T09:22:35.813 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T09:22:35.863 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:35 +0000 (0:00:00.982) 0:02:16.886 ******* 2019-12-09T09:22:36.648 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T09:22:36.649 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T09:22:36.650 INFO:teuthology.task.ansible.out: 2019-12-09T09:22:36.850 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T09:22:37.447 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T09:22:37.487 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:37 +0000 (0:00:01.623) 0:02:18.509 ******* 2019-12-09T09:22:37.729 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T09:22:37.804 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:22:37.848 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:37 +0000 (0:00:00.361) 0:02:18.870 ******* 2019-12-09T09:22:38.623 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi033.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T09:22:38.661 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:38 +0000 (0:00:00.813) 0:02:19.684 ******* 2019-12-09T09:22:38.689 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T09:22:38.690 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T09:22:38.975 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:22:39.013 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:39 +0000 (0:00:00.351) 0:02:20.036 ******* 2019-12-09T09:22:39.481 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T09:22:39.482 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T09:22:39.962 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T09:22:40.273 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T09:22:40.324 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:40 +0000 (0:00:01.310) 0:02:21.346 ******* 2019-12-09T09:22:40.354 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T09:22:40.356 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:40.396 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:40 +0000 (0:00:00.072) 0:02:21.418 ******* 2019-12-09T09:22:40.433 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T09:22:40.433 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:22:40.474 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:40 +0000 (0:00:00.078) 0:02:21.496 ******* 2019-12-09T09:22:40.798 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-09T09:22:40.799 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T09:22:40.836 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:40 +0000 (0:00:00.361) 0:02:21.858 ******* 2019-12-09T09:22:40.863 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T09:22:40.863 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T09:22:40.874 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T09:22:40.876 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:22:40.920 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:22:40.955 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:40 +0000 (0:00:00.119) 0:02:21.978 ******* 2019-12-09T09:22:41.743 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T09:22:41.744 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T09:22:41.787 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:41 +0000 (0:00:00.831) 0:02:22.809 ******* 2019-12-09T09:22:41.813 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T09:22:41.815 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:41.851 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:41 +0000 (0:00:00.064) 0:02:22.874 ******* 2019-12-09T09:22:41.888 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T09:22:41.889 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:22:41.925 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:41 +0000 (0:00:00.074) 0:02:22.948 ******* 2019-12-09T09:22:42.229 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-09T09:22:42.231 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T09:22:42.267 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:42 +0000 (0:00:00.341) 0:02:23.290 ******* 2019-12-09T09:22:42.314 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T09:22:42.316 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:22:42.325 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:22:42.343 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:22:42.354 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T09:22:42.391 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:42 +0000 (0:00:00.123) 0:02:23.414 ******* 2019-12-09T09:22:44.358 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-09T09:22:44.359 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T09:22:44.397 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:44 +0000 (0:00:02.005) 0:02:25.419 ******* 2019-12-09T09:22:44.424 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-09T09:22:44.425 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:44.462 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:44 +0000 (0:00:00.065) 0:02:25.485 ******* 2019-12-09T09:22:45.352 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-09T09:22:45.354 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T09:22:45.392 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:45 +0000 (0:00:00.929) 0:02:26.414 ******* 2019-12-09T09:22:45.827 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T09:22:45.868 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:45 +0000 (0:00:00.476) 0:02:26.891 ******* 2019-12-09T09:22:45.896 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-09T09:22:45.897 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:45.939 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:45 +0000 (0:00:00.070) 0:02:26.961 ******* 2019-12-09T09:22:46.323 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-09T09:22:46.362 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:46 +0000 (0:00:00.422) 0:02:27.384 ******* 2019-12-09T09:22:46.388 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-09T09:22:46.390 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:46.431 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:46 +0000 (0:00:00.069) 0:02:27.453 ******* 2019-12-09T09:22:46.749 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T09:22:46.750 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:22:46.801 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:46 +0000 (0:00:00.369) 0:02:27.823 ******* 2019-12-09T09:22:46.868 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-09T09:22:46.870 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T09:22:46.914 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:46 +0000 (0:00:00.113) 0:02:27.936 ******* 2019-12-09T09:22:47.177 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-09T09:22:47.179 INFO:teuthology.task.ansible.out:fatal: [smithi186.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T09:22:47.226 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:47 +0000 (0:00:00.311) 0:02:28.248 ******* 2019-12-09T09:22:47.262 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-09T09:22:47.264 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:47.318 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:47 +0000 (0:00:00.092) 0:02:28.340 ******* 2019-12-09T09:22:47.856 INFO:teuthology.task.ansible.out: 2019-12-09T09:22:47.857 INFO:teuthology.task.ansible.out:TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-09T09:22:47.896 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:47 +0000 (0:00:00.577) 0:02:28.918 ******* 2019-12-09T09:22:47.897 INFO:teuthology.task.ansible.out: 2019-12-09T09:22:47.925 INFO:teuthology.task.ansible.out: 2019-12-09T09:22:47.926 INFO:teuthology.task.ansible.out:TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:47.970 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:47 +0000 (0:00:00.074) 0:02:28.993 ******* 2019-12-09T09:22:47.972 INFO:teuthology.task.ansible.out: 2019-12-09T09:22:48.250 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-09T09:22:48.260 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T09:22:48.294 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:48 +0000 (0:00:00.323) 0:02:29.316 ******* 2019-12-09T09:22:48.311 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T09:22:48.312 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:22:48.351 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:48 +0000 (0:00:00.057) 0:02:29.374 ******* 2019-12-09T09:22:48.721 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-09T09:22:48.723 INFO:teuthology.task.ansible.out:fatal: [smithi033.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T09:22:48.758 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:48 +0000 (0:00:00.406) 0:02:29.780 ******* 2019-12-09T09:22:48.795 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:22:48.830 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:48 +0000 (0:00:00.071) 0:02:29.852 ******* 2019-12-09T09:22:49.182 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-09T09:22:49.185 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 09:20:07 UTC", "ActiveEnterTimestampMonotonic": "19195639", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "time-sync.target basic.target local-fs.target network.target nss-lookup.target systemd-journald.socket var-run.mount system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 09:20:07 UTC", "AssertTimestampMonotonic": "19195070", "Before": "multi-user.target xdm.service getty@tty1.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 09:20:07 UTC", "ConditionTimestampMonotonic": "19195068", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8977", "ExecMainStartTimestamp": "Mon 2019-12-09 09:20:07 UTC", "ExecMainStartTimestampMonotonic": "19195591", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-09T09:22:49.186 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-12-09 09:20:07 UTC] ; stop_time=[n/a] ; pid=8977 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 09:20:07 UTC", "InactiveExitTimestampMonotonic": "19195639", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8977", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target network-online.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-09T09:22:49.188 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 09:20:07 UTC", "WatchdogTimestampMonotonic": "19195621", "WatchdogUSec": "0"}} 2019-12-09T09:22:49.222 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:49 +0000 (0:00:00.392) 0:02:30.245 ******* 2019-12-09T09:22:49.530 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-09T09:22:49.531 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004868", "end": "2019-12-09 09:22:49.503091", "rc": 0, "start": "2019-12-09 09:22:49.498223", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T09:22:49.574 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:49 +0000 (0:00:00.352) 0:02:30.597 ******* 2019-12-09T09:22:49.792 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-09T09:22:49.794 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T09:22:49.839 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:49 +0000 (0:00:00.264) 0:02:30.862 ******* 2019-12-09T09:22:50.053 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-09T09:22:50.055 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T09:22:50.090 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:50 +0000 (0:00:00.251) 0:02:31.113 ******* 2019-12-09T09:22:50.285 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 09:19:57 UTC", "ActiveEnterTimestampMonotonic": "19661694", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target systemd-journald.socket time-sync.target local-fs.target basic.target var-run.mount system.slice network.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 09:19:57 UTC", "AssertTimestampMonotonic": "19661115", "Before": "xdm.service multi-user.target getty@tty1.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 09:19:57 UTC", "ConditionTimestampMonotonic": "19661112", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8760", "ExecMainStartTimestamp": "Mon 2019-12-09 09:19:57 UTC", "ExecMainStartTimestampMonotonic": "19661639", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbi 2019-12-09T09:22:50.287 INFO:teuthology.task.ansible.out:n/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Mon 2019-12-09 09:19:57 UTC] ; stop_time=[n/a] ; pid=8760 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 09:19:57 UTC", "InactiveExitTimestampMonotonic": "19661694", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8760", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", 2019-12-09T09:22:50.288 INFO:teuthology.task.ansible.out: "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 09:19:57 UTC", "WatchdogTimestampMonotonic": "19661674", "WatchdogUSec": "0"}} 2019-12-09T09:22:50.329 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:50 +0000 (0:00:00.232) 0:02:31.346 ******* 2019-12-09T09:22:50.662 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-09T09:22:50.663 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004632", "end": "2019-12-09 09:22:50.637624", "rc": 0, "start": "2019-12-09 09:22:50.632992", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T09:22:50.704 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:50 +0000 (0:00:00.381) 0:02:31.727 ******* 2019-12-09T09:22:51.231 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-09T09:22:51.232 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T09:22:51.269 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:22:51 +0000 (0:00:00.564) 0:02:32.291 ******* 2019-12-09T09:23:18.216 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-09T09:23:18.217 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-09T09:23:18.218 INFO:teuthology.task.ansible.out: 20 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T09:23:18.255 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:18 +0000 (0:00:26.985) 0:02:59.277 ******* 2019-12-09T09:23:18.943 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-12-09T09:23:18.945 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T09:23:18.981 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:18 +0000 (0:00:00.726) 0:03:00.004 ******* 2019-12-09T09:23:20.074 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-09T09:23:20.075 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.fileplanet.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-09T09:23:20.076 INFO:teuthology.task.ansible.out: 24 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T09:23:20.113 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:20 +0000 (0:00:01.131) 0:03:01.135 ******* 2019-12-09T09:23:20.671 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T09:23:20.712 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:20 +0000 (0:00:00.599) 0:03:01.735 ******* 2019-12-09T09:23:20.714 INFO:teuthology.task.ansible.out: 2019-12-09T09:23:27.008 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.742151", "end": "2019-12-09 09:23:26.963246", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:23:19.221095", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T09:23:27.050 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:27 +0000 (0:00:06.337) 0:03:08.073 ******* 2019-12-09T09:23:28.029 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2019-12-09T09:23:28.029 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T09:23:28.068 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:28 +0000 (0:00:01.017) 0:03:09.091 ******* 2019-12-09T09:23:28.422 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.019047", "end": "2019-12-09 09:23:28.390731", "rc": 0, "start": "2019-12-09 09:23:28.371684", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T09:23:28.465 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:28 +0000 (0:00:00.396) 0:03:09.487 ******* 2019-12-09T09:23:28.795 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-09T09:23:28.796 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.016672", "end": "2019-12-09 09:23:28.773084", "rc": 0, "start": "2019-12-09 09:23:28.756412", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:23:28.833 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:28 +0000 (0:00:00.368) 0:03:09.856 ******* 2019-12-09T09:23:29.603 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.570681", "end": "2019-12-09 09:23:29.571989", "failed_when_result": false, "rc": 0, "start": "2019-12-09 09:23:21.001308", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T09:23:29.644 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:29 +0000 (0:00:00.810) 0:03:10.666 ******* 2019-12-09T09:23:30.576 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2019-12-09T09:23:30.577 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T09:23:30.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:30 +0000 (0:00:00.970) 0:03:11.636 ******* 2019-12-09T09:23:30.938 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-09T09:23:30.938 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.017950", "end": "2019-12-09 09:23:30.911038", "rc": 0, "start": "2019-12-09 09:23:30.893088", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T09:23:30.986 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:30 +0000 (0:00:00.372) 0:03:12.008 ******* 2019-12-09T09:23:31.321 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-09T09:23:31.323 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.011900", "end": "2019-12-09 09:23:31.291782", "rc": 0, "start": "2019-12-09 09:23:31.279882", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:23:31.384 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:31 +0000 (0:00:00.397) 0:03:12.406 ******* 2019-12-09T09:23:36.771 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-09T09:23:36.773 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.668132", "end": "2019-12-09 09:23:36.730068", "rc": 0, "start": "2019-12-09 09:23:29.061936", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:23:36.816 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:36 +0000 (0:00:05.432) 0:03:17.839 ******* 2019-12-09T09:23:37.172 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-09T09:23:37.173 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T09:23:37.228 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:37 +0000 (0:00:00.411) 0:03:18.250 ******* 2019-12-09T09:23:37.262 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T09:23:37.263 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:23:37.318 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:37 +0000 (0:00:00.090) 0:03:18.340 ******* 2019-12-09T09:23:37.355 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-09T09:23:37.357 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:23:37.395 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:37 +0000 (0:00:00.076) 0:03:18.416 ******* 2019-12-09T09:23:37.431 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T09:23:37.432 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T09:23:37.471 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:37 +0000 (0:00:00.076) 0:03:18.493 ******* 2019-12-09T09:23:37.508 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T09:23:37.509 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:23:37.548 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:37 +0000 (0:00:00.076) 0:03:18.570 ******* 2019-12-09T09:23:37.869 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:23:37.903 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:37 +0000 (0:00:00.355) 0:03:18.926 ******* 2019-12-09T09:23:38.214 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-09T09:23:38.216 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T09:23:38.258 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:38 +0000 (0:00:00.354) 0:03:19.281 ******* 2019-12-09T09:23:38.285 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-09T09:23:38.287 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:38.342 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:38 +0000 (0:00:00.083) 0:03:19.364 ******* 2019-12-09T09:23:38.369 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T09:23:38.370 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:38.410 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:38 +0000 (0:00:00.068) 0:03:19.433 ******* 2019-12-09T09:23:38.784 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-09T09:23:38.786 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T09:23:38.824 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:38 +0000 (0:00:00.413) 0:03:19.846 ******* 2019-12-09T09:23:39.808 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-09T09:23:39.808 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T09:23:39.847 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:39 +0000 (0:00:01.023) 0:03:20.869 ******* 2019-12-09T09:23:39.884 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T09:23:39.886 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi186"}, "changed": false} 2019-12-09T09:23:39.923 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:39 +0000 (0:00:00.076) 0:03:20.946 ******* 2019-12-09T09:23:40.140 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-09T09:23:40.141 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.418017", "end": "2019-12-09 09:23:40.109432", "rc": 0, "start": "2019-12-09 09:23:31.691415", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:23:40.215 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:40 +0000 (0:00:00.292) 0:03:21.238 ******* 2019-12-09T09:23:40.594 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T09:23:40.635 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:40 +0000 (0:00:00.419) 0:03:21.657 ******* 2019-12-09T09:23:40.665 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T09:23:40.666 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T09:23:40.709 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:40 +0000 (0:00:00.073) 0:03:21.731 ******* 2019-12-09T09:23:40.752 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-09T09:23:40.753 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:23:40.808 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:40 +0000 (0:00:00.099) 0:03:21.830 ******* 2019-12-09T09:23:40.856 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T09:23:40.857 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T09:23:40.859 INFO:teuthology.task.ansible.out: 2019-12-09T09:23:40.898 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:40 +0000 (0:00:00.090) 0:03:21.921 ******* 2019-12-09T09:23:40.926 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-09T09:23:40.927 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi186.front.sepia.ceph.com", "ansible_hostname": "smithi186", "ansible_nodename": "smithi186"}, "changed": true, "name": "smithi186"} 2019-12-09T09:23:40.966 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:40 +0000 (0:00:00.068) 0:03:21.989 ******* 2019-12-09T09:23:40.974 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T09:23:40.975 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T09:23:41.022 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:41 +0000 (0:00:00.055) 0:03:22.045 ******* 2019-12-09T09:23:41.344 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-09T09:23:41.346 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T09:23:41.383 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:41 +0000 (0:00:00.361) 0:03:22.406 ******* 2019-12-09T09:23:41.728 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-09T09:23:41.729 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T09:23:41.768 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:41 +0000 (0:00:00.384) 0:03:22.790 ******* 2019-12-09T09:23:41.809 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-09T09:23:41.810 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:41.864 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:41 +0000 (0:00:00.095) 0:03:22.886 ******* 2019-12-09T09:23:41.874 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-12-09T09:23:41.876 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T09:23:41.930 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:41 +0000 (0:00:00.066) 0:03:22.953 ******* 2019-12-09T09:23:41.938 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T09:23:41.939 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:41.977 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:41 +0000 (0:00:00.046) 0:03:23.000 ******* 2019-12-09T09:23:42.355 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi033.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T09:23:42.401 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:42 +0000 (0:00:00.423) 0:03:23.423 ******* 2019-12-09T09:23:42.824 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-09T09:23:42.826 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T09:23:42.864 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:42 +0000 (0:00:00.460) 0:03:23.886 ******* 2019-12-09T09:23:43.439 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-09T09:23:43.440 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T09:23:43.479 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:43 +0000 (0:00:00.615) 0:03:24.501 ******* 2019-12-09T09:23:43.516 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T09:23:43.518 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi033"}, "changed": false} 2019-12-09T09:23:43.553 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:43 +0000 (0:00:00.074) 0:03:24.576 ******* 2019-12-09T09:23:43.819 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-09T09:23:43.820 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T09:23:43.862 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:43 +0000 (0:00:00.308) 0:03:24.884 ******* 2019-12-09T09:23:44.108 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi033.front.sepia.ceph.com", "ansible_hostname": "smithi033", "ansible_nodename": "smithi033"}, "changed": true, "name": "smithi033"} 2019-12-09T09:23:44.170 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:44 +0000 (0:00:00.301) 0:03:25.186 ******* 2019-12-09T09:23:44.823 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-09T09:23:44.824 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T09:23:44.893 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:44 +0000 (0:00:00.729) 0:03:25.916 ******* 2019-12-09T09:23:45.124 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-12-09T09:23:45.125 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T09:23:45.177 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:45 +0000 (0:00:00.283) 0:03:26.199 ******* 2019-12-09T09:23:45.215 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:23:45.215 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:23:45.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:45 +0000 (0:00:00.079) 0:03:26.278 ******* 2019-12-09T09:23:45.612 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:23:45.613 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T09:23:45.650 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:45 +0000 (0:00:00.394) 0:03:26.673 ******* 2019-12-09T09:23:46.032 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:23:46.033 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T09:23:46.107 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:46 +0000 (0:00:00.456) 0:03:27.130 ******* 2019-12-09T09:23:46.125 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-09T09:23:46.126 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T09:23:46.173 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:46 +0000 (0:00:00.065) 0:03:27.196 ******* 2019-12-09T09:23:47.028 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-09T09:23:47.028 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T09:23:47.031 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:23:47.031 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T09:23:47.071 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:47 +0000 (0:00:00.897) 0:03:28.094 ******* 2019-12-09T09:23:47.111 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:47 +0000 (0:00:00.039) 0:03:28.133 ******* 2019-12-09T09:23:48.000 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-09T09:23:48.000 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T09:23:48.042 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:48 +0000 (0:00:00.931) 0:03:29.064 ******* 2019-12-09T09:23:48.112 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:23:48.112 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T09:23:48.153 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:48 +0000 (0:00:00.111) 0:03:29.176 ******* 2019-12-09T09:23:48.370 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:23:48.371 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:23:48.422 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:48 +0000 (0:00:00.268) 0:03:29.444 ******* 2019-12-09T09:23:48.750 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:23:48.751 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T09:23:48.809 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:48 +0000 (0:00:00.387) 0:03:29.832 ******* 2019-12-09T09:23:49.182 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:23:49.183 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T09:23:49.240 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:49 +0000 (0:00:00.416) 0:03:30.263 ******* 2019-12-09T09:23:50.145 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T09:23:50.190 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:50 +0000 (0:00:00.949) 0:03:31.212 ******* 2019-12-09T09:23:50.715 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:23:50.717 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.225963", "end": "2019-12-09 09:23:50.680858", "rc": 0, "start": "2019-12-09 09:23:48.454895", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:23:50.757 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:50 +0000 (0:00:00.567) 0:03:31.779 ******* 2019-12-09T09:23:50.794 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T09:23:50.836 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:50 +0000 (0:00:00.079) 0:03:31.859 ******* 2019-12-09T09:23:51.209 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T09:23:51.250 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:51 +0000 (0:00:00.413) 0:03:32.272 ******* 2019-12-09T09:23:51.789 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T09:23:51.792 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T09:23:52.764 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T09:23:52.807 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:52 +0000 (0:00:01.556) 0:03:33.829 ******* 2019-12-09T09:23:52.833 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-09T09:23:52.835 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:52.887 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:52 +0000 (0:00:00.080) 0:03:33.909 ******* 2019-12-09T09:23:52.935 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T09:23:52.936 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:23:52.938 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:23:52.982 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:52 +0000 (0:00:00.095) 0:03:34.005 ******* 2019-12-09T09:23:53.020 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-09T09:23:53.021 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:53.059 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:53 +0000 (0:00:00.076) 0:03:34.082 ******* 2019-12-09T09:23:53.097 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:23:53.099 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T09:23:53.138 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:53 +0000 (0:00:00.078) 0:03:34.160 ******* 2019-12-09T09:23:53.644 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-09T09:23:53.646 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T09:23:53.690 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:53 +0000 (0:00:00.552) 0:03:34.712 ******* 2019-12-09T09:23:53.697 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:23:53.699 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.108005", "end": "2019-12-09 09:23:53.655564", "rc": 0, "start": "2019-12-09 09:23:51.547559", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:23:53.746 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:53 +0000 (0:00:00.056) 0:03:34.768 ******* 2019-12-09T09:23:53.774 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T09:23:53.818 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:53 +0000 (0:00:00.072) 0:03:34.841 ******* 2019-12-09T09:23:54.355 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-09T09:23:54.356 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.331318", "end": "2019-12-09 09:23:54.313810", "rc": 0, "start": "2019-12-09 09:23:53.982492", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.ash.fastserv.com", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T09:23:54.401 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:54 +0000 (0:00:00.582) 0:03:35.423 ******* 2019-12-09T09:23:54.756 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T09:23:54.757 INFO:teuthology.task.ansible.out:fatal: [smithi186.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.037119", "end": "2019-12-09 09:23:54.733654", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:23:54.696535", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-09T09:23:54.798 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:54 +0000 (0:00:00.392) 0:03:35.816 ******* 2019-12-09T09:23:54.827 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T09:23:54.828 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T09:23:54.829 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-09T09:23:54.831 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:54.877 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:54 +0000 (0:00:00.083) 0:03:35.899 ******* 2019-12-09T09:23:55.720 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T09:23:55.721 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T09:23:55.768 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:55 +0000 (0:00:00.890) 0:03:36.790 ******* 2019-12-09T09:23:55.820 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-09T09:23:55.822 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:55.866 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:55 +0000 (0:00:00.097) 0:03:36.888 ******* 2019-12-09T09:23:55.906 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T09:23:55.924 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:23:55.934 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:23:55.998 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:55 +0000 (0:00:00.132) 0:03:37.020 ******* 2019-12-09T09:23:56.026 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-09T09:23:56.028 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:56.067 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:56 +0000 (0:00:00.069) 0:03:37.090 ******* 2019-12-09T09:23:56.104 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:23:56.106 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T09:23:56.162 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:56 +0000 (0:00:00.094) 0:03:37.184 ******* 2019-12-09T09:23:56.637 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-09T09:23:56.639 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T09:23:56.694 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:56 +0000 (0:00:00.523) 0:03:37.708 ******* 2019-12-09T09:23:57.302 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-09T09:23:57.303 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.330915", "end": "2019-12-09 09:23:57.262192", "rc": 0, "start": "2019-12-09 09:23:56.931277", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.fileplanet.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.fileplanet.com", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T09:23:57.340 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:57 +0000 (0:00:00.655) 0:03:38.363 ******* 2019-12-09T09:23:57.709 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T09:23:57.710 INFO:teuthology.task.ansible.out:fatal: [smithi033.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.034636", "end": "2019-12-09 09:23:57.683941", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:23:57.649305", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-09T09:23:57.748 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:57 +0000 (0:00:00.407) 0:03:38.770 ******* 2019-12-09T09:23:57.786 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-09T09:23:57.787 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:23:57.824 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:23:57 +0000 (0:00:00.076) 0:03:38.847 ******* 2019-12-09T09:24:03.541 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T09:24:03.543 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T09:24:03.544 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T09:24:03.548 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:24:03.588 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:03 +0000 (0:00:05.763) 0:03:44.610 ******* 2019-12-09T09:24:06.921 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T09:24:06.923 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T09:24:06.924 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:24:06.964 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:06 +0000 (0:00:03.375) 0:03:47.986 ******* 2019-12-09T09:24:06.982 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:24:06.984 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T09:24:07.026 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:07 +0000 (0:00:00.060) 0:03:48.047 ******* 2019-12-09T09:24:10.457 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T09:24:10.493 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:10 +0000 (0:00:03.468) 0:03:51.515 ******* 2019-12-09T09:24:25.517 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:24:25.519 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T09:24:25.520 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T09:24:25.522 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies 2019-12-09T09:24:25.523 INFO:teuthology.task.ansible.out:Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 23 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T09:24:25.525 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.e 2019-12-09T09:24:25.526 INFO:teuthology.task.ansible.out:l7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:24:25.582 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:25 +0000 (0:00:15.089) 0:04:06.605 ******* 2019-12-09T09:24:26.353 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T09:24:26.397 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:26 +0000 (0:00:00.814) 0:04:07.419 ******* 2019-12-09T09:24:26.956 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:24:26.958 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T09:24:27.019 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:27 +0000 (0:00:00.622) 0:04:08.042 ******* 2019-12-09T09:24:27.072 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:27.159 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:27 +0000 (0:00:00.136) 0:04:08.178 ******* 2019-12-09T09:24:27.626 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice basic.target dbus.service polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "iptables.service shutdown.target ebtables.service ip6tables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": " 2019-12-09T09:24:27.628 INFO:teuthology.task.ansible.out:yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPr 2019-12-09T09:24:27.629 INFO:teuthology.task.ansible.out:iority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:24:27.670 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:27 +0000 (0:00:00.513) 0:04:08.692 ******* 2019-12-09T09:24:28.089 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-09T09:24:28.091 INFO:teuthology.task.ansible.out:fatal: [smithi186.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T09:24:28.142 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:28 +0000 (0:00:00.470) 0:04:09.163 ******* 2019-12-09T09:24:28.948 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-09T09:24:28.950 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T09:24:28.993 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:28 +0000 (0:00:00.852) 0:04:10.015 ******* 2019-12-09T09:24:29.020 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T09:24:29.021 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.060 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.067) 0:04:10.083 ******* 2019-12-09T09:24:29.097 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T09:24:29.137 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.076) 0:04:10.159 ******* 2019-12-09T09:24:29.163 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.203 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.066) 0:04:10.226 ******* 2019-12-09T09:24:29.230 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T09:24:29.232 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.269 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.066) 0:04:10.292 ******* 2019-12-09T09:24:29.307 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-09T09:24:29.308 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.309 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.351 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.082) 0:04:10.374 ******* 2019-12-09T09:24:29.388 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-09T09:24:29.390 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.434 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.082) 0:04:10.457 ******* 2019-12-09T09:24:29.461 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T09:24:29.497 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.063) 0:04:10.520 ******* 2019-12-09T09:24:29.524 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-09T09:24:29.526 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.562 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.064) 0:04:10.585 ******* 2019-12-09T09:24:29.592 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T09:24:29.594 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.636 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.074) 0:04:10.659 ******* 2019-12-09T09:24:29.668 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-09T09:24:29.670 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.704 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.068) 0:04:10.727 ******* 2019-12-09T09:24:29.732 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.771 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.066) 0:04:10.793 ******* 2019-12-09T09:24:29.797 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-09T09:24:29.799 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.836 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.064) 0:04:10.858 ******* 2019-12-09T09:24:29.863 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:24:29.866 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.919 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.083) 0:04:10.942 ******* 2019-12-09T09:24:29.964 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:29.994 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:29 +0000 (0:00:00.074) 0:04:11.016 ******* 2019-12-09T09:24:30.011 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:24:30.012 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T09:24:30.014 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T09:24:30.015 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.fileplanet.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Re 2019-12-09T09:24:30.017 INFO:teuthology.task.ansible.out:solved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 14 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T09:24:30.018 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T09:24:30.053 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.056) 0:04:11.073 ******* 2019-12-09T09:24:30.066 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.099 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.048) 0:04:11.122 ******* 2019-12-09T09:24:30.127 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.162 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.062) 0:04:11.184 ******* 2019-12-09T09:24:30.189 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-09T09:24:30.191 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.226 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.063) 0:04:11.248 ******* 2019-12-09T09:24:30.251 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-09T09:24:30.253 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.290 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.063) 0:04:11.312 ******* 2019-12-09T09:24:30.315 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-09T09:24:30.317 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.355 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.065) 0:04:11.377 ******* 2019-12-09T09:24:30.396 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-09T09:24:30.397 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.433 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.078) 0:04:11.455 ******* 2019-12-09T09:24:30.466 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T09:24:30.468 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.503 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.069) 0:04:11.525 ******* 2019-12-09T09:24:30.529 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T09:24:30.531 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.573 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.070) 0:04:11.595 ******* 2019-12-09T09:24:30.610 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:24:30.611 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.653 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.079) 0:04:11.675 ******* 2019-12-09T09:24:30.679 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:24:30.681 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.724 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.071) 0:04:11.747 ******* 2019-12-09T09:24:30.764 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:24:30.765 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.802 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.077) 0:04:11.825 ******* 2019-12-09T09:24:30.828 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:24:30.830 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.889 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.086) 0:04:11.911 ******* 2019-12-09T09:24:30.896 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-09T09:24:30.897 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T09:24:30.944 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.049) 0:04:11.961 ******* TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T09:24:30.947 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:30.993 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:30 +0000 (0:00:00.054) 0:04:12.015 ******* 2019-12-09T09:24:31.020 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:31.060 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.066) 0:04:12.082 ******* 2019-12-09T09:24:31.096 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:24:31.097 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:31.152 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.092) 0:04:12.175 ******* 2019-12-09T09:24:31.179 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:24:31.180 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:31.219 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.066) 0:04:12.241 ******* 2019-12-09T09:24:31.248 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:24:31.250 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:31.288 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.069) 0:04:12.310 ******* 2019-12-09T09:24:31.315 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:31.385 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.097) 0:04:12.408 ******* 2019-12-09T09:24:31.425 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:24:31.427 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:31.477 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.092) 0:04:12.500 ******* 2019-12-09T09:24:31.491 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T09:24:31.528 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.049) 0:04:12.549 ******* 2019-12-09T09:24:31.555 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T09:24:31.557 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:31.611 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.084) 0:04:12.633 ******* 2019-12-09T09:24:31.820 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:24:31.821 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T09:24:31.867 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:31 +0000 (0:00:00.256) 0:04:12.889 ******* 2019-12-09T09:24:32.067 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-09T09:24:32.069 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service basic.target dbus.service system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ebtables.service ip6tables.service iptables.service ipset.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-09T09:24:32.070 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-09T09:24:32.071 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:24:32.107 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.239) 0:04:13.129 ******* 2019-12-09T09:24:32.201 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:24:32.248 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.137) 0:04:13.267 ******* 2019-12-09T09:24:32.512 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-09T09:24:32.514 INFO:teuthology.task.ansible.out:fatal: [smithi033.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T09:24:32.568 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.324) 0:04:13.591 ******* 2019-12-09T09:24:32.579 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:24:32.581 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T09:24:32.622 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.053) 0:04:13.644 ******* 2019-12-09T09:24:32.681 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:24:32.683 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:32.719 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.097) 0:04:13.741 ******* 2019-12-09T09:24:32.768 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:24:32.770 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:32.817 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.098) 0:04:13.840 ******* 2019-12-09T09:24:32.856 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:24:32.857 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:32.896 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.079) 0:04:13.919 ******* 2019-12-09T09:24:32.936 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:32.980 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:32 +0000 (0:00:00.083) 0:04:14.003 ******* 2019-12-09T09:24:32.990 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-09T09:24:32.996 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T09:24:33.040 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.059) 0:04:14.063 ******* 2019-12-09T09:24:33.048 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:24:33.049 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.103 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.062) 0:04:14.125 ******* 2019-12-09T09:24:33.108 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T09:24:33.110 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.150 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.045) 0:04:14.171 ******* 2019-12-09T09:24:33.155 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:24:33.156 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.194 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.045) 0:04:14.217 ******* 2019-12-09T09:24:33.200 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T09:24:33.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.056) 0:04:14.273 ******* 2019-12-09T09:24:33.258 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.296 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.045) 0:04:14.319 ******* 2019-12-09T09:24:33.302 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-09T09:24:33.303 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.346 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.049) 0:04:14.368 ******* 2019-12-09T09:24:33.352 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:24:33.354 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.405 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.058) 0:04:14.427 ******* 2019-12-09T09:24:33.413 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T09:24:33.414 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.467 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.056) 0:04:14.484 ******* 2019-12-09T09:24:33.468 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:24:33.470 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.514 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.052) 0:04:14.537 ******* 2019-12-09T09:24:33.519 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-09T09:24:33.520 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.521 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.562 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.047) 0:04:14.584 ******* 2019-12-09T09:24:33.569 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-09T09:24:33.570 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.051) 0:04:14.636 ******* 2019-12-09T09:24:33.621 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.663 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.050) 0:04:14.686 ******* 2019-12-09T09:24:33.669 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-09T09:24:33.673 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.710 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.046) 0:04:14.733 ******* 2019-12-09T09:24:33.716 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T09:24:33.762 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.050) 0:04:14.783 ******* 2019-12-09T09:24:33.768 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:24:33.770 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.820 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.058) 0:04:14.842 ******* 2019-12-09T09:24:33.827 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-09T09:24:33.828 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.881 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.061) 0:04:14.904 ******* 2019-12-09T09:24:33.889 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:24:33.891 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.931 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.049) 0:04:14.953 ******* 2019-12-09T09:24:33.939 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T09:24:33.941 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:33.976 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:33 +0000 (0:00:00.045) 0:04:14.999 ******* 2019-12-09T09:24:33.983 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-09T09:24:33.985 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.030 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.053) 0:04:15.053 ******* 2019-12-09T09:24:34.045 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.078 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.047) 0:04:15.100 ******* 2019-12-09T09:24:34.088 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.132 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.048) 0:04:15.149 ******* 2019-12-09T09:24:34.134 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.177 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.050) 0:04:15.199 ******* 2019-12-09T09:24:34.192 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.222 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.045) 0:04:15.244 ******* 2019-12-09T09:24:34.230 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.265 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.043) 0:04:15.288 ******* 2019-12-09T09:24:34.271 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-09T09:24:34.273 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.309 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.044) 0:04:15.332 ******* 2019-12-09T09:24:34.324 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.352 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.043) 0:04:15.375 ******* 2019-12-09T09:24:34.358 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T09:24:34.360 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.396 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.043) 0:04:15.419 ******* 2019-12-09T09:24:34.410 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-09T09:24:34.412 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.448 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.051) 0:04:15.471 ******* 2019-12-09T09:24:34.454 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-09T09:24:34.456 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.502 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.054) 0:04:15.525 ******* 2019-12-09T09:24:34.510 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-09T09:24:34.511 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.566 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.063) 0:04:15.588 ******* 2019-12-09T09:24:34.571 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-09T09:24:34.572 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.049) 0:04:15.637 ******* 2019-12-09T09:24:34.623 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.661 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.046) 0:04:15.683 ******* 2019-12-09T09:24:34.667 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-09T09:24:34.670 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.707 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.046) 0:04:15.729 ******* 2019-12-09T09:24:34.713 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-09T09:24:34.715 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.752 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.045) 0:04:15.775 ******* 2019-12-09T09:24:34.759 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-09T09:24:34.760 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.798 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.045) 0:04:15.821 ******* 2019-12-09T09:24:34.804 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-09T09:24:34.805 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.845 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.046) 0:04:15.867 ******* 2019-12-09T09:24:34.851 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:24:34.853 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T09:24:34.890 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.045) 0:04:15.913 ******* 2019-12-09T09:24:34.897 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-09T09:24:34.899 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.939 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:34 +0000 (0:00:00.048) 0:04:15.961 ******* 2019-12-09T09:24:34.946 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:24:34.948 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:34.959 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-09T09:24:34.978 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi186.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.019 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.080) 0:04:16.041 ******* 2019-12-09T09:24:35.025 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.027 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.028 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.029 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.071 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.051) 0:04:16.093 ******* 2019-12-09T09:24:35.078 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T09:24:35.080 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.116 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.045) 0:04:16.139 ******* 2019-12-09T09:24:35.144 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.191 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.074) 0:04:16.214 ******* 2019-12-09T09:24:35.221 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T09:24:35.224 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.264 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.073) 0:04:16.287 ******* 2019-12-09T09:24:35.279 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi186.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003571", "end": "2019-12-09 09:24:35.211914", "rc": 0, "start": "2019-12-09 09:24:35.208343", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T09:24:35.327 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.061) 0:04:16.349 ******* 2019-12-09T09:24:35.332 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T09:24:35.334 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.378 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.048) 0:04:16.398 ******* 2019-12-09T09:24:35.382 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T09:24:35.384 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.431 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.056) 0:04:16.454 ******* 2019-12-09T09:24:35.439 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T09:24:35.441 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.484 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.052) 0:04:16.507 ******* 2019-12-09T09:24:35.492 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.532 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.047) 0:04:16.554 ******* 2019-12-09T09:24:35.537 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T09:24:35.538 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.577 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.045) 0:04:16.600 ******* 2019-12-09T09:24:35.583 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-09T09:24:35.585 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.625 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.048) 0:04:16.648 ******* 2019-12-09T09:24:35.634 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.671 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.045) 0:04:16.693 ******* 2019-12-09T09:24:35.680 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-09T09:24:35.682 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.720 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.049) 0:04:16.743 ******* 2019-12-09T09:24:35.729 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.775 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.054) 0:04:16.798 ******* 2019-12-09T09:24:35.803 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T09:24:35.805 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.853 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.077) 0:04:16.875 ******* 2019-12-09T09:24:35.901 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T09:24:35.903 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:35.945 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:35 +0000 (0:00:00.091) 0:04:16.967 ******* 2019-12-09T09:24:35.997 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T09:24:35.999 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:36.051 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.106) 0:04:17.073 ******* 2019-12-09T09:24:36.090 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:36.129 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.077) 0:04:17.151 ******* 2019-12-09T09:24:36.166 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T09:24:36.167 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:36.208 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.079) 0:04:17.230 ******* 2019-12-09T09:24:36.217 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-09T09:24:36.219 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T09:24:36.259 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.050) 0:04:17.281 ******* 2019-12-09T09:24:36.309 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:24:36.309 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item={u'block_used': 13040458, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284026, u'size_available': 930955370496, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284026, "block_size": 4096, "block_total": 240324484, "block_used": 13040458, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955370496, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T09:24:36.350 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.090) 0:04:17.372 ******* 2019-12-09T09:24:36.537 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:24:36.539 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T09:24:36.575 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.225) 0:04:17.598 ******* 2019-12-09T09:24:36.685 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-09T09:24:36.687 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013892", "end": "2019-12-09 09:24:36.651779", "rc": 0, "start": "2019-12-09 09:24:36.637887", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:36.734 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.157) 0:04:17.756 ******* 2019-12-09T09:24:36.772 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-09T09:24:36.773 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'block_used': 13040458, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284026, u'size_available': 930955370496, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284026, "block_size": 4096, "block_total": 240324484, "block_used": 13040458, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955370496, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:36.822 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.087) 0:04:17.844 ******* 2019-12-09T09:24:36.848 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T09:24:36.887 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.065) 0:04:17.909 ******* 2019-12-09T09:24:36.925 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:24:36.926 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTX2', u'wwn-0x5000c5009293ee14'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XTX2', u'wwn': u'0x5000c5009293ee14', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTX2-part1', u'wwn-0x5000c5009293ee14-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTX2", "wwn-0x5000c5009293ee14"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTX2-part1", "wwn-0x5000c5009293ee14-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTX2", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:24:36.927 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c5009293ee14"}}, "skip_reason": "Conditional result was False"} TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:24:36.929 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:24:36.983 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:36 +0000 (0:00:00.096) 0:04:18.006 ******* 2019-12-09T09:24:37.316 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:24:37.317 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T09:24:37.361 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.377) 0:04:18.384 ******* 2019-12-09T09:24:37.399 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:24:37.401 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.438 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.076) 0:04:18.460 ******* 2019-12-09T09:24:37.466 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:24:37.473 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.524 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.083) 0:04:18.543 ******* 2019-12-09T09:24:37.549 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.585 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.064) 0:04:18.608 ******* 2019-12-09T09:24:37.613 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:24:37.614 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.665 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.079) 0:04:18.687 ******* 2019-12-09T09:24:37.695 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.754 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.089) 0:04:18.777 ******* 2019-12-09T09:24:37.783 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:24:37.786 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.828 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.069) 0:04:18.847 ******* 2019-12-09T09:24:37.852 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T09:24:37.854 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.895 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.070) 0:04:18.917 ******* 2019-12-09T09:24:37.921 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T09:24:37.924 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:37.975 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:37 +0000 (0:00:00.080) 0:04:18.998 ******* 2019-12-09T09:24:38.003 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T09:24:38.004 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.045 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.069) 0:04:19.067 ******* 2019-12-09T09:24:38.071 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-09T09:24:38.073 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.115 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.070) 0:04:19.137 ******* 2019-12-09T09:24:38.142 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-09T09:24:38.144 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.183 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.068) 0:04:19.205 ******* 2019-12-09T09:24:38.210 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T09:24:38.212 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.254 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.071) 0:04:19.277 ******* 2019-12-09T09:24:38.282 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T09:24:38.284 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.326 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.072) 0:04:19.349 ******* 2019-12-09T09:24:38.371 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-09T09:24:38.373 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.412 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.085) 0:04:19.434 ******* 2019-12-09T09:24:38.449 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-09T09:24:38.451 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.495 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.083) 0:04:19.518 ******* 2019-12-09T09:24:38.534 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-09T09:24:38.543 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.592 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.096) 0:04:19.614 ******* 2019-12-09T09:24:38.620 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-09T09:24:38.621 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.660 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.068) 0:04:19.683 ******* 2019-12-09T09:24:38.688 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T09:24:38.689 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.738 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.077) 0:04:19.760 ******* 2019-12-09T09:24:38.765 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-09T09:24:38.767 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.805 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.067) 0:04:19.828 ******* 2019-12-09T09:24:38.833 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-09T09:24:38.834 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.884 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.078) 0:04:19.907 ******* 2019-12-09T09:24:38.905 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-ebbmyR-GHIM-QKy8-7dgh-CUjT-nWNU-aQzIz7', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300CB400BGN', u'nvme-nvme.8086-43564654363233333030434234303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.631879", "end": "2019-12-09 09:24:38.841261", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-ebbmyR-GHIM-QKy8-7dgh-CUjT-nWNU-aQzIz7", "nvme-INTEL_SSDPEDMD400G4_CVFT623300CB400BGN", "nvme-nvme.8086-43564654363233333030434234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "372.61 GB", "support_discard 2019-12-09T09:24:38.907 INFO:teuthology.task.ansible.out:": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:37.209382", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:38.922 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:38.956 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:38 +0000 (0:00:00.071) 0:04:19.978 ******* 2019-12-09T09:24:38.990 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-09T09:24:38.992 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.030 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.074) 0:04:20.052 ******* 2019-12-09T09:24:39.067 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:24:39.069 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T09:24:39.108 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.077) 0:04:20.130 ******* 2019-12-09T09:24:39.157 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T09:24:39.159 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.160 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-09T09:24:39.171 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.182 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.193 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.204 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.206 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.247 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.139) 0:04:20.269 ******* 2019-12-09T09:24:39.421 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-09T09:24:39.423 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004227", "end": "2019-12-09 09:24:39.384427", "rc": 0, "start": "2019-12-09 09:24:39.380200", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T09:24:39.463 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.215) 0:04:20.485 ******* 2019-12-09T09:24:39.522 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T09:24:39.523 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.567 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.102) 0:04:20.588 ******* 2019-12-09T09:24:39.606 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-09T09:24:39.607 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.650 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.083) 0:04:20.672 ******* 2019-12-09T09:24:39.709 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-09T09:24:39.711 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.748 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.098) 0:04:20.771 ******* 2019-12-09T09:24:39.777 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-09T09:24:39.779 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:39.834 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:39 +0000 (0:00:00.085) 0:04:20.856 ******* 2019-12-09T09:24:40.304 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:24:40.306 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1Rdzh19tMmH53AREFjKUvJValsdKXvMNq9'], u'uuids': [u'fb0943a1-3d8e-4e12-92ac-1d1c0fb1144f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.062752", "end": "2019-12-09 09:24:40.251349", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1Rdzh19tMmH53AREFjKUvJValsdKXvMNq9"], "labels": [], "masters": [], "uuids": ["fb0943a1-3d8e-4e12-92ac-1d1c0fb1144f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:39.188597", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:40.339 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-09T09:24:40.340 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T09:24:40.389 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:40 +0000 (0:00:00.555) 0:04:21.412 ******* 2019-12-09T09:24:40.441 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi033.front.sepia.ceph.com] => (item={u'block_used': 13040463, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284021, u'size_available': 930955350016, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284021, "block_size": 4096, "block_total": 240324484, "block_used": 13040463, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955350016, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T09:24:40.496 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:40 +0000 (0:00:00.098) 0:04:21.510 ******* 2019-12-09T09:24:40.848 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013974", "end": "2019-12-09 09:24:40.816299", "rc": 0, "start": "2019-12-09 09:24:40.802325", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:40.888 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:40 +0000 (0:00:00.400) 0:04:21.911 ******* 2019-12-09T09:24:40.939 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-09T09:24:40.941 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'block_used': 13040463, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284021, u'size_available': 930955350016, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284021, "block_size": 4096, "block_total": 240324484, "block_used": 13040463, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955350016, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:40.979 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:40 +0000 (0:00:00.090) 0:04:22.002 ******* 2019-12-09T09:24:41.006 INFO:teuthology.task.ansible.out: 2019-12-09T09:24:41.008 INFO:teuthology.task.ansible.out:TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T09:24:41.054 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:41 +0000 (0:00:00.074) 0:04:22.076 ******* 2019-12-09T09:24:41.056 INFO:teuthology.task.ansible.out: 2019-12-09T09:24:41.119 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:24:41.121 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2L', u'wwn-0x5000c500871a8992'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4HQ2L', u'wwn': u'0x5000c500871a8992', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2L-part1', u'wwn-0x5000c500871a8992-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2L", "wwn-0x5000c500871a8992"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2L-part1", "wwn-0x5000c500871a8992-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQ2L", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:24:41.123 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871a8992"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:41.696 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R9oo1GpbfhsBLhS5mfFz3J8W07M1Oj0IX'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.079732", "end": "2019-12-09 09:24:41.663425", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R9oo1GpbfhsBLhS5mfFz3J8W07M1Oj0IX"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:40.583693", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:42.931 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-tfmoe3-uxIN-2z3f-c8bt-Ycz5-ZdTt-om3ZEs', u'nvme-INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN', u'nvme-nvme.8086-43564654353333303030385134303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.455537", "end": "2019-12-09 09:24:42.886859", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-tfmoe3-uxIN-2z3f-c8bt-Ycz5-ZdTt-om3ZEs", "nvme-INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN", "nvme-nvme.8086-43564654353333303030385134303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:41.431 2019-12-09T09:24:42.932 INFO:teuthology.task.ansible.out:322", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:43.102 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R8TTw0HA8JH1BFyj2Np1uOgQ32dBXtvTJ'], u'uuids': [u'73d0b544-3320-4701-9f66-cde2580f2c10']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.073836", "end": "2019-12-09 09:24:43.059746", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R8TTw0HA8JH1BFyj2Np1uOgQ32dBXtvTJ"], "labels": [], "masters": [], "uuids": ["73d0b544-3320-4701-9f66-cde2580f2c10"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:41.985910", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:44.321 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX4L85IXPDtEw2tMiWAagLGS3R2p480g3E'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.059027", "end": "2019-12-09 09:24:44.280491", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX4L85IXPDtEw2tMiWAagLGS3R2p480g3E"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:43.221464", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:44.490 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RbfGatL82xJfsImDLHIkWH5vDR6pekW0l'], u'uuids': [u'cf2e1240-007d-4cfc-8cdb-9413726435ef']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.069779", "end": "2019-12-09 09:24:44.448507", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RbfGatL82xJfsImDLHIkWH5vDR6pekW0l"], "labels": [], "masters": [], "uuids": ["cf2e1240-007d-4cfc-8cdb-9413726435ef"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:43.378728", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:45.707 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXf5PUYux353Osai7Pnh3H5V5Wo1w724qz'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.059508", "end": "2019-12-09 09:24:45.671936", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXf5PUYux353Osai7Pnh3H5V5Wo1w724qz"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:44.612428", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:45.867 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RwpJI2aKvAj44yKP0wuG9ogJ2TO7eJ4eE'], u'uuids': [u'5943f491-ae8f-40dd-9912-904ab2ad051b']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.064585", "end": "2019-12-09 09:24:45.834307", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RwpJI2aKvAj44yKP0wuG9ogJ2TO7eJ4eE"], "labels": [], "masters": [], "uuids": ["5943f491-ae8f-40dd-9912-904ab2ad051b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:44.769722", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:45.938 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:45 +0000 (0:00:04.884) 0:04:26.961 ******* 2019-12-09T09:24:46.000 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:46.001 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTX2', u'wwn-0x5000c5009293ee14'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XTX2', u'wwn': u'0x5000c5009293ee14', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XTX2-part1', u'wwn-0x5000c5009293ee14-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTX2", "wwn-0x5000c5009293ee14"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XTX2-part1", "wwn-0x5000c5009293ee14-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XTX2", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:24:46.002 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c5009293ee14"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:46.355 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-ebbmyR-GHIM-QKy8-7dgh-CUjT-nWNU-aQzIz7', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300CB400BGN', u'nvme-nvme.8086-43564654363233333030434234303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.064318", "end": "2019-12-09 09:24:46.320963", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-ebbmyR-GHIM-QKy8-7dgh-CUjT-nWNU-aQzIz7", "nvme-INTEL_SSDPEDMD400G4_CVFT623300CB400BGN", "nvme-nvme.8086-43564654363233333030434234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:46.256645", "stderr": 2019-12-09T09:24:46.356 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0578887 s, 2.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0578887 s, 2.0 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:46.843 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1Rdzh19tMmH53AREFjKUvJValsdKXvMNq9'], u'uuids': [u'fb0943a1-3d8e-4e12-92ac-1d1c0fb1144f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.202706", "end": "2019-12-09 09:24:46.809391", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1Rdzh19tMmH53AREFjKUvJValsdKXvMNq9"], "labels": [], "masters": [], "uuids": ["fb0943a1-3d8e-4e12-92ac-1d1c0fb1144f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:46.606685", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.196174 s, 588 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.196174 s, 588 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:47.154 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:24:47.155 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX38ERLRKe3KnwsYl1VG4K4dMppEYnBkkJ'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.108702", "end": "2019-12-09 09:24:47.114896", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX38ERLRKe3KnwsYl1VG4K4dMppEYnBkkJ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:46.006194", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:47.375 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:47.376 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R9oo1GpbfhsBLhS5mfFz3J8W07M1Oj0IX'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.205191", "end": "2019-12-09 09:24:47.343576", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R9oo1GpbfhsBLhS5mfFz3J8W07M1Oj0IX"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:47.138385", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.198538 s, 581 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.198538 s, 581 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:47.915 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R8TTw0HA8JH1BFyj2Np1uOgQ32dBXtvTJ'], u'uuids': [u'73d0b544-3320-4701-9f66-cde2580f2c10']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.207412", "end": "2019-12-09 09:24:47.876146", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1R8TTw0HA8JH1BFyj2Np1uOgQ32dBXtvTJ"], "labels": [], "masters": [], "uuids": ["73d0b544-3320-4701-9f66-cde2580f2c10"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:47.668734", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.20081 s, 574 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.20081 s, 574 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:48.400 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RbfGatL82xJfsImDLHIkWH5vDR6pekW0l'], u'uuids': [u'cf2e1240-007d-4cfc-8cdb-9413726435ef']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.185518", "end": "2019-12-09 09:24:48.361805", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RbfGatL82xJfsImDLHIkWH5vDR6pekW0l"], "labels": [], "masters": [], "uuids": ["cf2e1240-007d-4cfc-8cdb-9413726435ef"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:48.176287", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.179006 s, 644 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.179006 s, 644 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:48.562 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:24:48.563 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXtqe6QdizQpHlJokj7zU0QpC6dtwThA8n'], u'uuids': [u'71bf5525-5be6-4d86-a581-02a879519ca7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.056695", "end": "2019-12-09 09:24:48.491140", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXtqe6QdizQpHlJokj7zU0QpC6dtwThA8n"], "labels": [], "masters": [], "uuids": ["71bf5525-5be6-4d86-a581-02a879519ca7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:47.434445", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:48.862 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:48.863 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RwpJI2aKvAj44yKP0wuG9ogJ2TO7eJ4eE'], u'uuids': [u'5943f491-ae8f-40dd-9912-904ab2ad051b']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT623300CB400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.171360", "end": "2019-12-09 09:24:48.823224", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-eaFvV8NZNtStoWt5oonuHavXjYfw1g1RwpJI2aKvAj44yKP0wuG9ogJ2TO7eJ4eE"], "labels": [], "masters": [], "uuids": ["5943f491-ae8f-40dd-9912-904ab2ad051b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT623300CB400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:48.651864", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.16516 s, 698 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.16516 s, 698 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:48.908 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:48 +0000 (0:00:02.969) 0:04:29.930 ******* 2019-12-09T09:24:49.333 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-09T09:24:49.335 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.083079", "end": "2019-12-09 09:24:49.286088", "rc": 0, "start": "2019-12-09 09:24:49.203009", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:49.374 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:49 +0000 (0:00:00.466) 0:04:30.396 ******* 2019-12-09T09:24:49.807 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T09:24:49.809 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.076259", "end": "2019-12-09 09:24:49.762467", "rc": 0, "start": "2019-12-09 09:24:49.686208", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:49.873 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:49 +0000 (0:00:00.499) 0:04:30.896 ******* 2019-12-09T09:24:49.905 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T09:24:49.958 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:49 +0000 (0:00:00.085) 0:04:30.981 ******* 2019-12-09T09:24:49.965 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T09:24:49.967 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXAPlmGZqbChrayWX04kmtkn6cHk3te5LB'], u'uuids': [u'609f5803-a0ef-47f3-bb5e-7e0e15ee7e1b']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.051558", "end": "2019-12-09 09:24:49.889430", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXAPlmGZqbChrayWX04kmtkn6cHk3te5LB"], "labels": [], "masters": [], "uuids": ["609f5803-a0ef-47f3-bb5e-7e0e15ee7e1b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:48.837872", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T09:24:50.020 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.062) 0:04:31.043 ******* 2019-12-09T09:24:50.026 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-09T09:24:50.028 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.071 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.050) 0:04:31.093 ******* 2019-12-09T09:24:50.077 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:50.079 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2L', u'wwn-0x5000c500871a8992'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4HQ2L', u'wwn': u'0x5000c500871a8992', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2L-part1', u'wwn-0x5000c500871a8992-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2L", "wwn-0x5000c500871a8992"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2L-part1", "wwn-0x5000c500871a8992-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQ2L", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T09:24:50.081 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871a8992"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.102 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-09T09:24:50.104 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.154 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.082) 0:04:31.176 ******* 2019-12-09T09:24:50.191 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:24:50.193 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.231 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.077) 0:04:31.254 ******* 2019-12-09T09:24:50.258 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-09T09:24:50.260 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.301 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.068) 0:04:31.322 ******* 2019-12-09T09:24:50.489 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-tfmoe3-uxIN-2z3f-c8bt-Ycz5-ZdTt-om3ZEs', u'nvme-INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN', u'nvme-nvme.8086-43564654353333303030385134303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.086922", "end": "2019-12-09 09:24:50.442427", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-tfmoe3-uxIN-2z3f-c8bt-Ycz5-ZdTt-om3ZEs", "nvme-INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN", "nvme-nvme.8086-43564654353333303030385134303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "372.61 GB", "support_discard": "512", "vend 2019-12-09T09:24:50.492 INFO:teuthology.task.ansible.out:or": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:50.355505", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0801759 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0801759 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:50.622 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-09T09:24:50.624 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:24:50.663 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.363) 0:04:31.686 ******* 2019-12-09T09:24:50.707 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:24:50.708 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={u'block_used': 13040458, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284026, u'size_available': 930955370496, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284026, "block_size": 4096, "block_total": 240324484, "block_used": 13040458, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955370496, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.748 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.084) 0:04:31.771 ******* 2019-12-09T09:24:50.786 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-09T09:24:50.786 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.824 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.075) 0:04:31.847 ******* 2019-12-09T09:24:50.853 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:24:50.855 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.906 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.081) 0:04:31.928 ******* 2019-12-09T09:24:50.933 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:24:50.935 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:50.976 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:50 +0000 (0:00:00.070) 0:04:31.999 ******* 2019-12-09T09:24:51.003 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:51.004 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX4L85IXPDtEw2tMiWAagLGS3R2p480g3E'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.196975", "end": "2019-12-09 09:24:50.964569", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX4L85IXPDtEw2tMiWAagLGS3R2p480g3E"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:50.767594", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.190552 s, 605 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.190552 s, 605 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:51.018 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-09T09:24:51.019 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:51.059 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:51 +0000 (0:00:00.083) 0:04:32.082 ******* 2019-12-09T09:24:51.087 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T09:24:51.088 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:51.133 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:51 +0000 (0:00:00.073) 0:04:32.155 ******* 2019-12-09T09:24:51.182 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-09T09:24:51.183 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:51.243 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:51 +0000 (0:00:00.110) 0:04:32.265 ******* 2019-12-09T09:24:51.502 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:51.503 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXf5PUYux353Osai7Pnh3H5V5Wo1w724qz'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.195900", "end": "2019-12-09 09:24:51.468903", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXf5PUYux353Osai7Pnh3H5V5Wo1w724qz"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:51.273003", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.188961 s, 610 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.188961 s, 610 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:52.013 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX38ERLRKe3KnwsYl1VG4K4dMppEYnBkkJ'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.199319", "end": "2019-12-09 09:24:51.976012", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLX38ERLRKe3KnwsYl1VG4K4dMppEYnBkkJ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:51.776693", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.192842 s, 598 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.192842 s, 598 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:52.108 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-09T09:24:52.110 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T09:24:52.148 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:52 +0000 (0:00:00.905) 0:04:33.171 ******* 2019-12-09T09:24:52.497 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:52.498 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXtqe6QdizQpHlJokj7zU0QpC6dtwThA8n'], u'uuids': [u'71bf5525-5be6-4d86-a581-02a879519ca7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.191095", "end": "2019-12-09 09:24:52.467600", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXtqe6QdizQpHlJokj7zU0QpC6dtwThA8n"], "labels": [], "masters": [], "uuids": ["71bf5525-5be6-4d86-a581-02a879519ca7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:52.276505", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.184626 s, 625 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.184626 s, 625 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:52.940 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:53.026 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T09:24:53.028 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXAPlmGZqbChrayWX04kmtkn6cHk3te5LB'], u'uuids': [u'609f5803-a0ef-47f3-bb5e-7e0e15ee7e1b']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'CVFT5330008Q400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.210210", "end": "2019-12-09 09:24:52.994004", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-17pHw5vr63sGVzsGtoNtgWuvC4wtJiLXAPlmGZqbChrayWX04kmtkn6cHk3te5LB"], "labels": [], "masters": [], "uuids": ["609f5803-a0ef-47f3-bb5e-7e0e15ee7e1b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "CVFT5330008Q400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 09:24:52.783794", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.203692 s, 566 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.203692 s, 566 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:53.103 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:53 +0000 (0:00:00.954) 0:04:34.126 ******* 2019-12-09T09:24:53.428 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:24:53.430 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:53.541 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-09T09:24:53.543 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.082812", "end": "2019-12-09 09:24:53.494617", "rc": 0, "start": "2019-12-09 09:24:53.411805", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:53.580 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:53 +0000 (0:00:00.476) 0:04:34.603 ******* 2019-12-09T09:24:53.916 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:24:53.917 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:53.975 INFO:teuthology.task.ansible.out: 2019-12-09T09:24:53.977 INFO:teuthology.task.ansible.out:TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T09:24:53.991 INFO:teuthology.task.ansible.out: 2019-12-09T09:24:53.993 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.051436", "end": "2019-12-09 09:24:53.940911", "rc": 0, "start": "2019-12-09 09:24:53.889475", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:24:54.035 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.454) 0:04:35.058 ******* 2019-12-09T09:24:54.062 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T09:24:54.104 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.068) 0:04:35.126 ******* 2019-12-09T09:24:54.157 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-09T09:24:54.159 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:54.212 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.108) 0:04:35.234 ******* 2019-12-09T09:24:54.250 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-09T09:24:54.252 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:54.292 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.080) 0:04:35.315 ******* 2019-12-09T09:24:54.336 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:24:54.338 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:54.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.112) 0:04:35.427 ******* 2019-12-09T09:24:54.411 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:24:54.412 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:54.436 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:54.480 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.075) 0:04:35.502 ******* 2019-12-09T09:24:54.831 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:24:54.837 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi033.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T09:24:54.902 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.421) 0:04:35.924 ******* 2019-12-09T09:24:54.945 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.043) 0:04:35.968 ******* 2019-12-09T09:24:54.950 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T09:24:54.951 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={u'block_used': 13040463, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284021, u'size_available': 930955350016, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966950, u'device': u'/dev/sda1', u'inode_used': 88026, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284021, "block_size": 4096, "block_total": 240324484, "block_used": 13040463, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966950, "inode_total": 61054976, "inode_used": 88026, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955350016, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:54.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:54 +0000 (0:00:00.049) 0:04:36.017 ******* 2019-12-09T09:24:55.034 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-09T09:24:55.036 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:55.093 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:55 +0000 (0:00:00.098) 0:04:36.116 ******* 2019-12-09T09:24:55.136 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:24:55.138 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:55.189 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:55 +0000 (0:00:00.095) 0:04:36.212 ******* 2019-12-09T09:24:55.221 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:24:55.223 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:55.287 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:55 +0000 (0:00:00.097) 0:04:36.309 ******* 2019-12-09T09:24:55.297 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:24:55.303 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T09:24:55.347 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:55 +0000 (0:00:00.060) 0:04:36.370 ******* 2019-12-09T09:24:55.357 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:55.413 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:55 +0000 (0:00:00.065) 0:04:36.436 ******* 2019-12-09T09:24:55.417 INFO:teuthology.task.ansible.out: 2019-12-09T09:24:55.426 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** skipping: [smithi186.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:55.460 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T09:24:55.462 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:55.508 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:55 +0000 (0:00:00.095) 0:04:36.531 ******* 2019-12-09T09:24:55.563 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-09T09:24:55.565 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:55.604 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:55 +0000 (0:00:00.095) 0:04:36.627 ******* 2019-12-09T09:24:55.723 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:24:55.724 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:24:56.023 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:24:56.181 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-09T09:24:56.182 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T09:24:56.228 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:56 +0000 (0:00:00.623) 0:04:37.250 ******* 2019-12-09T09:24:56.332 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:24:56.340 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:24:56.634 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:24:56.680 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:56 +0000 (0:00:00.451) 0:04:37.702 ******* 2019-12-09T09:24:56.772 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:24:56.774 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:57.018 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:24:57.081 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:57 +0000 (0:00:00.401) 0:04:38.103 ******* 2019-12-09T09:24:57.124 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T09:24:57.183 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:57 +0000 (0:00:00.101) 0:04:38.205 ******* 2019-12-09T09:24:57.302 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T09:24:57.304 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:57.826 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:58.324 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:58.819 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T09:24:58.873 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:58 +0000 (0:00:01.690) 0:04:39.896 ******* 2019-12-09T09:24:58.911 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T09:24:58.913 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true} 2019-12-09T09:24:58.957 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:58 +0000 (0:00:00.083) 0:04:39.979 ******* 2019-12-09T09:24:59.218 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T09:24:59.220 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T09:24:59.286 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:59 +0000 (0:00:00.324) 0:04:40.304 ******* 2019-12-09T09:24:59.341 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:24:59.342 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:59.664 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:24:59.758 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T09:24:59.799 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:59 +0000 (0:00:00.517) 0:04:40.821 ******* 2019-12-09T09:24:59.828 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:59.888 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:59 +0000 (0:00:00.089) 0:04:40.910 ******* 2019-12-09T09:24:59.917 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:24:59.962 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:24:59 +0000 (0:00:00.073) 0:04:40.984 ******* 2019-12-09T09:24:59.994 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:00.045 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.082) 0:04:41.067 ******* 2019-12-09T09:25:00.052 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:25:00.066 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:00.109 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.064) 0:04:41.132 ******* 2019-12-09T09:25:00.140 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:00.182 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.072) 0:04:41.205 ******* 2019-12-09T09:25:00.220 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:00.258 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.076) 0:04:41.281 ******* 2019-12-09T09:25:00.295 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-09T09:25:00.297 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:00.344 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.080) 0:04:41.361 ******* 2019-12-09T09:25:00.349 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T09:25:00.350 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:25:00.690 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T09:25:00.758 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.418) 0:04:41.780 ******* 2019-12-09T09:25:00.831 INFO:teuthology.task.ansible.out: 2019-12-09T09:25:00.833 INFO:teuthology.task.ansible.out:TASK [testnode : Install ntp package on rpm based systems.] ******************** ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-09T09:25:00.885 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.128) 0:04:41.908 ******* 2019-12-09T09:25:00.923 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-09T09:25:00.924 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:00.964 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:00 +0000 (0:00:00.078) 0:04:41.987 ******* 2019-12-09T09:25:00.991 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-09T09:25:00.993 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:01.030 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:01 +0000 (0:00:00.065) 0:04:42.052 ******* 2019-12-09T09:25:01.107 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-09T09:25:01.109 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T09:25:01.154 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:01 +0000 (0:00:00.124) 0:04:42.177 ******* 2019-12-09T09:25:01.193 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-09T09:25:01.195 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T09:25:01.234 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:01 +0000 (0:00:00.079) 0:04:42.256 ******* 2019-12-09T09:25:01.692 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T09:25:01.694 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true} 2019-12-09T09:25:01.735 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:01 +0000 (0:00:00.501) 0:04:42.757 ******* 2019-12-09T09:25:01.978 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-09T09:25:01.978 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T09:25:02.018 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.283) 0:04:43.041 ******* 2019-12-09T09:25:02.045 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T09:25:02.047 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.086 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.067) 0:04:43.108 ******* 2019-12-09T09:25:02.304 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-09T09:25:02.306 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T09:25:02.343 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.256) 0:04:43.365 ******* 2019-12-09T09:25:02.373 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-12-09T09:25:02.374 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.412 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.069) 0:04:43.435 ******* 2019-12-09T09:25:02.439 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-09T09:25:02.440 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.496 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.072) 0:04:43.508 ******* 2019-12-09T09:25:02.513 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-09T09:25:02.515 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.558 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.073) 0:04:43.581 ******* 2019-12-09T09:25:02.600 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-09T09:25:02.602 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.667 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.108) 0:04:43.689 ******* 2019-12-09T09:25:02.679 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-09T09:25:02.682 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sntp.service basic.target systemd-journald.socket -.mount ntpdate.service syslog.target tmp.mount system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target chronyd.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T09:25:02.684 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T09:25:02.685 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:25:02.740 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.072) 0:04:43.762 ******* 2019-12-09T09:25:02.747 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-09T09:25:02.749 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.787 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.047) 0:04:43.809 ******* 2019-12-09T09:25:02.825 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-12-09T09:25:02.827 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.868 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.081) 0:04:43.890 ******* 2019-12-09T09:25:02.895 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-09T09:25:02.897 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:02.946 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:02 +0000 (0:00:00.075) 0:04:43.966 ******* 2019-12-09T09:25:03.458 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-09T09:25:03.459 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-09T09:25:03.501 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:03 +0000 (0:00:00.557) 0:04:44.524 ******* 2019-12-09T09:25:03.529 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-09T09:25:03.531 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:03.580 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:03 +0000 (0:00:00.079) 0:04:44.603 ******* 2019-12-09T09:25:03.610 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-09T09:25:03.612 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:03.681 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:03 +0000 (0:00:00.100) 0:04:44.704 ******* 2019-12-09T09:25:03.693 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-09T09:25:03.695 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T09:25:03.763 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:03 +0000 (0:00:00.081) 0:04:44.785 ******* 2019-12-09T09:25:03.793 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-09T09:25:03.794 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:03.832 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:03 +0000 (0:00:00.069) 0:04:44.855 ******* 2019-12-09T09:25:03.859 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2019-12-09T09:25:03.861 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:03.899 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:03 +0000 (0:00:00.066) 0:04:44.921 ******* 2019-12-09T09:25:04.236 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-09T09:25:04.237 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-09T09:25:04.285 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:04 +0000 (0:00:00.385) 0:04:45.307 ******* 2019-12-09T09:25:04.707 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T09:25:04.766 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:04 +0000 (0:00:00.480) 0:04:45.788 ******* 2019-12-09T09:25:04.795 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T09:25:04.797 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:04.848 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:04 +0000 (0:00:00.081) 0:04:45.870 ******* 2019-12-09T09:25:05.269 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-12-09T09:25:05.271 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T09:25:05.320 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:05 +0000 (0:00:00.469) 0:04:46.340 ******* 2019-12-09T09:25:06.393 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-09T09:25:06.394 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "syslog.target basic.target ntpdate.service systemd-journald.socket -.mount tmp.mount sntp.service system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T09:25:06.396 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T09:25:06.397 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T09:25:06.438 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:06 +0000 (0:00:01.120) 0:04:47.460 ******* 2019-12-09T09:25:07.156 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-12-09T09:25:07.157 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-09T09:25:07.201 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.763) 0:04:48.224 ******* 2019-12-09T09:25:07.230 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-09T09:25:07.232 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:07.268 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.066) 0:04:48.290 ******* 2019-12-09T09:25:07.296 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-09T09:25:07.298 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-09T09:25:07.340 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.069) 0:04:48.360 ******* 2019-12-09T09:25:07.345 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-09T09:25:07.346 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T09:25:07.385 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.047) 0:04:48.407 ******* 2019-12-09T09:25:07.412 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-09T09:25:07.413 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:07.479 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.094) 0:04:48.502 ******* 2019-12-09T09:25:07.519 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2019-12-09T09:25:07.522 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:07.564 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.084) 0:04:48.586 ******* 2019-12-09T09:25:07.646 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-09T09:25:07.648 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T09:25:07.686 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.122) 0:04:48.709 ******* 2019-12-09T09:25:07.715 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-09T09:25:07.716 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:07.753 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.067) 0:04:48.776 ******* 2019-12-09T09:25:07.780 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-09T09:25:07.782 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T09:25:07.828 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.074) 0:04:48.851 ******* 2019-12-09T09:25:07.856 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-09T09:25:07.858 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:07.895 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.067) 0:04:48.918 ******* 2019-12-09T09:25:07.901 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-09T09:25:07.902 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-09T09:25:07.953 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:07 +0000 (0:00:00.057) 0:04:48.975 ******* 2019-12-09T09:25:08.219 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-09T09:25:08.221 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T09:25:08.276 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:08 +0000 (0:00:00.323) 0:04:49.299 ******* 2019-12-09T09:25:08.594 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-09T09:25:08.596 INFO:teuthology.task.ansible.out:changed: [smithi186.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T09:25:08.653 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:08 +0000 (0:00:00.376) 0:04:49.675 ******* 2019-12-09T09:25:08.692 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-09T09:25:08.694 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.733 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:08 +0000 (0:00:00.079) 0:04:49.755 ******* 2019-12-09T09:25:08.770 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-09T09:25:08.779 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.790 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.793 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.804 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.864 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:08 +0000 (0:00:00.131) 0:04:49.886 ******* 2019-12-09T09:25:08.890 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-12-09T09:25:08.891 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T09:25:08.945 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:08 +0000 (0:00:00.082) 0:04:49.968 ******* 2019-12-09T09:25:08.953 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-09T09:25:08.953 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.955 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.956 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi186.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:08.958 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.003 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:08 +0000 (0:00:00.057) 0:04:50.025 ******* 2019-12-09T09:25:09.029 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T09:25:09.031 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.077 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.074) 0:04:50.100 ******* 2019-12-09T09:25:09.104 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T09:25:09.106 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.141 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.064) 0:04:50.164 ******* 2019-12-09T09:25:09.168 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T09:25:09.170 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.208 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.066) 0:04:50.231 ******* 2019-12-09T09:25:09.245 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-09T09:25:09.247 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.300 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.089) 0:04:50.320 ******* 2019-12-09T09:25:09.324 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-09T09:25:09.326 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.364 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.066) 0:04:50.386 ******* 2019-12-09T09:25:09.391 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.431 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.067) 0:04:50.454 ******* 2019-12-09T09:25:09.461 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.511 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.080) 0:04:50.534 ******* 2019-12-09T09:25:09.539 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T09:25:09.540 INFO:teuthology.task.ansible.out:skipping: [smithi186.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:09.578 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:09 +0000 (0:00:00.066) 0:04:50.600 ******* 2019-12-09T09:25:09.929 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:25:09.930 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.013053", "end": "2019-12-09 09:25:09.886651", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 09:25:09.873598", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:10.245 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.008012", "end": "2019-12-09 09:25:10.208676", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 09:25:10.200664", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:10.444 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-12-09T09:25:10.446 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-09T09:25:10.488 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:10 +0000 (0:00:00.910) 0:04:51.511 ******* 2019-12-09T09:25:10.524 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:10.553 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:10 +0000 (0:00:00.064) 0:04:51.576 ******* 2019-12-09T09:25:10.560 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:25:10.560 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007877", "end": "2019-12-09 09:25:10.527792", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:25:10.519915", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:10.582 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-09T09:25:10.584 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2019-12-09T09:25:10.628 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:10 +0000 (0:00:00.071) 0:04:51.648 ******* 2019-12-09T09:25:10.859 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:25:10.859 INFO:teuthology.task.ansible.out:ok: [smithi186.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007409", "end": "2019-12-09 09:25:10.816714", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 09:25:10.809305", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:10.906 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:10 +0000 (0:00:00.274) 0:04:51.922 ******* 2019-12-09T09:25:10.925 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T09:25:10.972 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:10 +0000 (0:00:00.072) 0:04:51.994 ******* 2019-12-09T09:25:10.990 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi033.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T09:25:11.039 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:11 +0000 (0:00:00.067) 0:04:52.062 ******* 2019-12-09T09:25:11.066 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-09T09:25:11.068 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:11.121 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:11 +0000 (0:00:00.081) 0:04:52.143 ******* 2019-12-09T09:25:11.148 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-09T09:25:11.149 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T09:25:11.192 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:11 +0000 (0:00:00.070) 0:04:52.214 ******* 2019-12-09T09:25:11.219 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-09T09:25:11.220 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:11.255 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:11 +0000 (0:00:00.062) 0:04:52.277 ******* 2019-12-09T09:25:11.264 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi186.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:25:11.587 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi033.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T09:25:11.620 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:11 +0000 (0:00:00.365) 0:04:52.643 ******* 2019-12-09T09:25:11.959 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-09T09:25:11.960 INFO:teuthology.task.ansible.out:changed: [smithi033.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T09:25:11.999 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:11 +0000 (0:00:00.378) 0:04:53.021 ******* 2019-12-09T09:25:12.025 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-09T09:25:12.026 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.068 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.069) 0:04:53.091 ******* 2019-12-09T09:25:12.118 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-09T09:25:12.120 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.121 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.132 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.133 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.134 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.175 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.105) 0:04:53.197 ******* 2019-12-09T09:25:12.212 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-09T09:25:12.213 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.215 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.225 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.226 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.237 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.277 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.102) 0:04:53.299 ******* 2019-12-09T09:25:12.303 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.350 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.072) 0:04:53.372 ******* 2019-12-09T09:25:12.376 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T09:25:12.378 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.414 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.063) 0:04:53.436 ******* 2019-12-09T09:25:12.440 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T09:25:12.442 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.492 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.077) 0:04:53.514 ******* 2019-12-09T09:25:12.524 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-09T09:25:12.526 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.564 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.072) 0:04:53.587 ******* 2019-12-09T09:25:12.591 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-09T09:25:12.592 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.637 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.073) 0:04:53.660 ******* 2019-12-09T09:25:12.678 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.718 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.080) 0:04:53.740 ******* 2019-12-09T09:25:12.746 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.788 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.069) 0:04:53.810 ******* 2019-12-09T09:25:12.814 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T09:25:12.815 INFO:teuthology.task.ansible.out:skipping: [smithi033.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T09:25:12.851 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:12 +0000 (0:00:00.063) 0:04:53.874 ******* 2019-12-09T09:25:13.219 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T09:25:13.220 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.013433", "end": "2019-12-09 09:25:13.169987", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 09:25:13.156554", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:13.533 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007313", "end": "2019-12-09 09:25:13.500073", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 09:25:13.492760", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:13.868 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007301", "end": "2019-12-09 09:25:13.831284", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 09:25:13.823983", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:14.183 INFO:teuthology.task.ansible.out:ok: [smithi033.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008745", "end": "2019-12-09 09:25:14.151197", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 09:25:14.142452", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T09:25:14.222 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:14 +0000 (0:00:01.370) 0:04:55.244 ******* 2019-12-09T09:25:14.248 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T09:25:14.291 INFO:teuthology.task.ansible.out:Monday 09 December 2019 09:25:14 +0000 (0:00:00.068) 0:04:55.313 ******* 2019-12-09T09:25:14.653 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi033.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T09:25:14.768 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2019-12-09T09:25:14.772 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2019-12-09T09:25:14.775 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2019-12-09T09:25:14.777 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi033.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 smithi186.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 Monday 09 December 2019 09:25:14 +0000 (0:00:00.484) 0:04:55.798 ******* =============================================================================== 2019-12-09T09:25:14.778 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 35.36s users : Update authorized_keys using the keys repo --------------------- 34.64s common : nrpe - Ensure SELinux policy is up to date -------------------- 26.99s users : Remove revoked users ------------------------------------------- 15.75s testnode : Install packages -------------------------------------------- 15.09s common : Update yum cache ---------------------------------------------- 11.99s common : nrpe - Remove SELinux policy package --------------------------- 6.34s testnode : Ensure ceph packages are not present. ------------------------ 5.76s common : nrpe - Load SELinux policy package ----------------------------- 5.43s 2019-12-09T09:25:14.780 INFO:teuthology.task.ansible.out:testnode : Zap all non-root disks --------------------------------------- 4.88s Gathering Facts --------------------------------------------------------- 3.34s common : Make sure smartmontools is installed --------------------------- 3.19s ansible-managed : Add authorized keys for the ansible user. ------------- 2.83s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 2.01s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.62s testnode : Rebuild rpmdb ------------------------------------------------ 1.56s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.48s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.37s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 1.31s common : Configure local epel mirrorlists ------------------------------- 1.09s 2019-12-09T09:25:15.064 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi186.front.sepia.ceph.com')] 2019-12-09T09:25:15.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:25:15.067 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi033.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:25:15.167 INFO:teuthology.orchestra.run.smithi033:> true 2019-12-09T09:25:15.561 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T09:25:15.562 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi186.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T09:25:15.661 INFO:teuthology.orchestra.run.smithi186:> true 2019-12-09T09:25:16.045 INFO:teuthology.run_tasks:Running task clock... 2019-12-09T09:25:16.171 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-12-09T09:25:16.172 INFO:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T09:25:16.312 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T09:25:26.403 INFO:teuthology.orchestra.run.smithi033.stdout:ntpd: time slew -0.000782s 2019-12-09T09:25:26.441 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T09:25:26.524 INFO:teuthology.orchestra.run.smithi033.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T09:25:26.524 INFO:teuthology.orchestra.run.smithi033.stdout:============================================================================== 2019-12-09T09:25:26.524 INFO:teuthology.orchestra.run.smithi033.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:26.524 INFO:teuthology.orchestra.run.smithi033.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:26.524 INFO:teuthology.orchestra.run.smithi033.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:26.525 INFO:teuthology.orchestra.run.smithi033.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:26.525 INFO:teuthology.orchestra.run.smithi186:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T09:25:26.667 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T09:25:36.758 INFO:teuthology.orchestra.run.smithi186.stdout:ntpd: time slew +0.000232s 2019-12-09T09:25:36.798 INFO:teuthology.orchestra.run.smithi186.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T09:25:36.871 INFO:teuthology.orchestra.run.smithi186.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T09:25:36.871 INFO:teuthology.orchestra.run.smithi186.stdout:============================================================================== 2019-12-09T09:25:36.871 INFO:teuthology.orchestra.run.smithi186.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:36.871 INFO:teuthology.orchestra.run.smithi186.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:36.872 INFO:teuthology.orchestra.run.smithi186.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:36.872 INFO:teuthology.orchestra.run.smithi186.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T09:25:36.873 INFO:teuthology.run_tasks:Running task install... 2019-12-09T09:25:36.886 DEBUG:teuthology.task.install:project ceph 2019-12-09T09:25:36.886 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}} 2019-12-09T09:25:36.886 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'} 2019-12-09T09:25:36.886 INFO:teuthology.task.install:Using flavor: notcmalloc 2019-12-09T09:25:36.891 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-daemon', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-daemon', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2019-12-09T09:25:36.891 INFO:teuthology.task.install:extra packages: [] 2019-12-09T09:25:36.891 INFO:teuthology.orchestra.run.smithi033:> sudo lsb_release -is 2019-12-09T09:25:37.059 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS 2019-12-09T09:25:37.064 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:25:37.064 INFO:teuthology.orchestra.run.smithi186:> sudo lsb_release -is 2019-12-09T09:25:37.065 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T09:25:37.065 INFO:teuthology.orchestra.run.smithi033:> sudo lsb_release -is 2019-12-09T09:25:37.196 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS 2019-12-09T09:25:37.197 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:25:37.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:25:37.233 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS 2019-12-09T09:25:37.234 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:25:37.235 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T09:25:37.235 INFO:teuthology.orchestra.run.smithi186:> sudo lsb_release -is 2019-12-09T09:25:37.362 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T09:25:37.362 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T09:25:37.372 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS 2019-12-09T09:25:37.374 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:25:37.375 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:25:37.480 INFO:teuthology.orchestra.run.smithi033:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T09:25:37.544 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T09:25:37.544 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T09:25:37.602 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T09:25:37.602 INFO:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T09:25:37.669 INFO:teuthology.orchestra.run.smithi186:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T09:25:37.706 INFO:teuthology.orchestra.run.smithi033:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T09:25:37.790 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T09:25:37.790 INFO:teuthology.orchestra.run.smithi186:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T09:25:37.809 INFO:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T09:25:37.893 INFO:teuthology.orchestra.run.smithi186:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T09:25:37.900 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2019-12-09T09:25:37.902 INFO:teuthology.orchestra.run.smithi033:> sudo yum clean all 2019-12-09T09:25:37.980 INFO:teuthology.orchestra.run.smithi186:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T09:25:38.054 INFO:teuthology.orchestra.run.smithi186.stdout:check_obsoletes = 1 2019-12-09T09:25:38.056 INFO:teuthology.orchestra.run.smithi186:> sudo yum clean all 2019-12-09T09:25:38.103 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:25:38.143 INFO:teuthology.orchestra.run.smithi033.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T09:25:38.143 INFO:teuthology.orchestra.run.smithi033.stdout: : lab-extras updates 2019-12-09T09:25:38.189 INFO:teuthology.orchestra.run.smithi033.stdout:Cleaning up list of fastest mirrors 2019-12-09T09:25:38.190 INFO:teuthology.orchestra.run.smithi033.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T09:25:38.203 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw 2019-12-09T09:25:38.217 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:25:38.255 INFO:teuthology.orchestra.run.smithi186.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T09:25:38.255 INFO:teuthology.orchestra.run.smithi186.stdout: : lab-extras updates 2019-12-09T09:25:38.302 INFO:teuthology.orchestra.run.smithi186.stdout:Cleaning up list of fastest mirrors 2019-12-09T09:25:38.302 INFO:teuthology.orchestra.run.smithi186.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T09:25:38.315 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-radosgw 2019-12-09T09:25:38.408 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:25:38.445 INFO:teuthology.orchestra.run.smithi033.stdout:Determining fastest mirrors 2019-12-09T09:25:38.487 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:25:38.524 INFO:teuthology.orchestra.run.smithi186.stdout:Determining fastest mirrors 2019-12-09T09:25:38.905 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:25:38.908 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:25:38.908 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:25:38.909 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:25:39.069 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:25:39.070 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:25:39.070 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:25:39.071 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:25:46.690 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:25:46.891 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:25:47.053 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:25:47.053 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:25:47.054 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.067 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.178 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.179 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.180 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.183 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.183 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.184 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.185 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.187 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.188 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:25:47.188 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.196 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.198 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.199 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.200 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.202 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.202 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.205 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T09:25:47.206 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T09:25:47.206 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:25:47.206 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:25:47.208 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T09:25:47.209 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.211 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:25:47.211 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.215 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.216 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.218 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.219 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.219 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.221 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.221 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.222 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.223 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.224 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.225 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.225 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T09:25:47.226 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.226 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:25:47.226 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:25:47.228 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T09:25:47.228 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:25:47.229 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:25:47.230 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:25:47.230 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T09:25:47.230 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.231 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.231 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.231 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.233 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T09:25:47.233 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.233 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.234 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.235 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T09:25:47.244 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:25:47.245 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:25:47.245 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.259 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.366 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.386 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.386 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.388 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.388 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.388 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.390 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.390 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.390 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:25:47.390 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.390 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.390 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.397 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.397 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.397 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.397 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.398 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T09:25:47.399 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T09:25:47.399 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:25:47.399 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:25:47.399 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T09:25:47.399 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.400 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:25:47.400 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.403 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.404 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.405 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.406 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.407 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.408 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.408 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.409 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.410 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.411 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.412 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:25:47.412 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T09:25:47.413 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.413 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T09:25:47.413 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T09:25:47.414 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T09:25:47.414 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:25:47.416 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T09:25:47.416 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:25:47.416 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T09:25:47.417 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.418 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.418 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.418 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.420 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T09:25:47.420 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.420 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.420 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:25:47.422 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T09:25:48.139 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:25:48.229 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:25:48.229 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:25:48.232 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:25:48.233 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:25:48.233 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository 2019-12-09T09:25:48.233 INFO:teuthology.orchestra.run.smithi033.stdout: Size 2019-12-09T09:25:48.233 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:25:48.233 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:25:48.233 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T09:25:48.234 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:25:48.234 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T09:25:48.234 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T09:25:48.234 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T09:25:48.234 INFO:teuthology.orchestra.run.smithi033.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T09:25:48.234 INFO:teuthology.orchestra.run.smithi033.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T09:25:48.234 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T09:25:48.235 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T09:25:48.235 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T09:25:48.235 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T09:25:48.235 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T09:25:48.235 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T09:25:48.235 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T09:25:48.236 INFO:teuthology.orchestra.run.smithi033.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T09:25:48.236 INFO:teuthology.orchestra.run.smithi033.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T09:25:48.236 INFO:teuthology.orchestra.run.smithi033.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T09:25:48.236 INFO:teuthology.orchestra.run.smithi033.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T09:25:48.236 INFO:teuthology.orchestra.run.smithi033.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T09:25:48.236 INFO:teuthology.orchestra.run.smithi033.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T09:25:48.237 INFO:teuthology.orchestra.run.smithi033.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T09:25:48.237 INFO:teuthology.orchestra.run.smithi033.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T09:25:48.237 INFO:teuthology.orchestra.run.smithi033.stdout:Updating for dependencies: 2019-12-09T09:25:48.237 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T09:25:48.237 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T09:25:48.237 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:25:48.237 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:25:48.238 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:25:48.238 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T09:25:48.238 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T09:25:48.238 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:25:48.238 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 45 M 2019-12-09T09:25:48.238 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:25:48.239 INFO:teuthology.orchestra.run.smithi033.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:25:48.294 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:25:48.381 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:25:48.381 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:25:48.385 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:25:48.385 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:25:48.385 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository 2019-12-09T09:25:48.385 INFO:teuthology.orchestra.run.smithi186.stdout: Size 2019-12-09T09:25:48.385 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:25:48.385 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:25:48.386 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T09:25:48.386 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:25:48.386 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T09:25:48.386 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T09:25:48.386 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T09:25:48.386 INFO:teuthology.orchestra.run.smithi186.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T09:25:48.387 INFO:teuthology.orchestra.run.smithi186.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T09:25:48.387 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T09:25:48.387 INFO:teuthology.orchestra.run.smithi186.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T09:25:48.387 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T09:25:48.387 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T09:25:48.388 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T09:25:48.388 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T09:25:48.388 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T09:25:48.388 INFO:teuthology.orchestra.run.smithi186.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T09:25:48.388 INFO:teuthology.orchestra.run.smithi186.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T09:25:48.389 INFO:teuthology.orchestra.run.smithi186.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T09:25:48.389 INFO:teuthology.orchestra.run.smithi186.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T09:25:48.389 INFO:teuthology.orchestra.run.smithi186.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T09:25:48.389 INFO:teuthology.orchestra.run.smithi186.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T09:25:48.389 INFO:teuthology.orchestra.run.smithi186.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T09:25:48.389 INFO:teuthology.orchestra.run.smithi186.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T09:25:48.389 INFO:teuthology.orchestra.run.smithi186.stdout:Updating for dependencies: 2019-12-09T09:25:48.390 INFO:teuthology.orchestra.run.smithi186.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T09:25:48.390 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T09:25:48.390 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:25:48.390 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:25:48.390 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:25:48.390 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T09:25:48.390 INFO:teuthology.orchestra.run.smithi186.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T09:25:48.391 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:25:48.391 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 45 M 2019-12-09T09:25:48.391 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:25:48.391 INFO:teuthology.orchestra.run.smithi186.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T09:25:51.252 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:25:51.252 INFO:teuthology.orchestra.run.smithi033.stdout:Total 15 MB/s | 45 MB 00:03 2019-12-09T09:25:51.269 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:25:51.343 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:25:51.588 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:25:51.588 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:25:52.361 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T09:25:52.704 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:25:52.704 INFO:teuthology.orchestra.run.smithi186.stdout:Total 10 MB/s | 45 MB 00:04 2019-12-09T09:25:52.713 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:25:52.756 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:25:52.861 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T09:25:52.961 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:25:52.962 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:25:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T09:25:53.523 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:25:53.713 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T09:25:53.780 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:25:54.081 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T09:25:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T09:25:54.438 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T09:25:54.817 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:25:55.116 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:25:55.254 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T09:25:55.598 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T09:25:55.642 INFO:teuthology.orchestra.run.smithi033.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T09:25:56.514 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:25:56.533 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T09:25:56.881 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:25:57.245 INFO:teuthology.orchestra.run.smithi186.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T09:25:57.398 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T09:25:57.756 INFO:teuthology.orchestra.run.smithi033.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:25:58.034 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T09:25:58.160 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:25:58.301 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T09:25:58.528 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:25:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:25:58.892 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:25:59.137 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T09:25:59.205 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T09:25:59.532 INFO:teuthology.orchestra.run.smithi186.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:25:59.687 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T09:25:59.837 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T09:26:00.164 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T09:26:00.526 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:26:00.857 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:26:01.141 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T09:26:01.518 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T09:26:02.160 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T09:26:03.355 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:26:03.923 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T09:26:03.948 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T09:26:05.141 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:26:05.740 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T09:26:14.088 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:26:14.452 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:26:14.734 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:26:15.061 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:26:15.191 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T09:26:15.325 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T09:26:15.405 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:26:15.458 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T09:26:15.559 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:26:15.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:26:15.802 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T09:26:15.873 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:26:15.886 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T09:26:16.030 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T09:26:16.120 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:26:16.145 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:26:16.204 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:26:16.363 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T09:26:16.496 INFO:teuthology.orchestra.run.smithi186.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:26:16.496 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:26:16.595 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T09:26:16.656 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T09:26:16.712 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T09:26:16.764 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T09:26:16.821 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T09:26:16.865 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:26:16.947 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T09:26:16.998 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:26:17.072 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T09:26:17.124 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T09:26:17.197 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T09:26:17.242 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T09:26:17.339 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T09:26:17.375 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T09:26:17.465 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T09:26:17.484 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:26:17.607 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T09:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T09:26:17.693 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:26:17.730 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T09:26:17.786 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:26:17.787 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:26:17.866 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T09:26:18.009 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T09:26:18.110 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:26:18.110 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:18.110 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:26:18.110 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.110 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:18.111 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:26:18.111 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.111 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.111 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.111 INFO:teuthology.orchestra.run.smithi033.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T09:26:18.111 INFO:teuthology.orchestra.run.smithi033.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T09:26:18.111 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.112 INFO:teuthology.orchestra.run.smithi033.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T09:26:18.112 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T09:26:18.112 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.112 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T09:26:18.112 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.112 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T09:26:18.113 INFO:teuthology.orchestra.run.smithi033.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.113 INFO:teuthology.orchestra.run.smithi033.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.113 INFO:teuthology.orchestra.run.smithi033.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.113 INFO:teuthology.orchestra.run.smithi033.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T09:26:18.113 INFO:teuthology.orchestra.run.smithi033.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.113 INFO:teuthology.orchestra.run.smithi033.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.113 INFO:teuthology.orchestra.run.smithi033.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.114 INFO:teuthology.orchestra.run.smithi033.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T09:26:18.114 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:18.114 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Updated: 2019-12-09T09:26:18.114 INFO:teuthology.orchestra.run.smithi033.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.114 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:18.114 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:18.114 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:26:18.128 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T09:26:18.232 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-test 2019-12-09T09:26:18.252 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T09:26:18.344 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T09:26:18.461 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T09:26:18.512 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:18.552 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:18.553 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:18.554 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:18.554 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:18.554 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:18.564 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T09:26:18.679 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T09:26:18.762 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T09:26:18.898 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T09:26:19.005 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T09:26:19.147 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T09:26:19.230 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:19.298 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T09:26:19.299 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T09:26:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:26:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:19.579 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:19.592 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:19.623 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:19.624 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:19.625 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:19.625 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T09:26:19.626 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T09:26:19.627 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T09:26:19.629 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T09:26:19.631 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:19.632 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T09:26:19.714 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T09:26:19.714 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:19.714 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:26:19.714 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.714 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:19.715 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:26:19.715 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.715 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.715 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.715 INFO:teuthology.orchestra.run.smithi186.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T09:26:19.715 INFO:teuthology.orchestra.run.smithi186.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T09:26:19.715 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.716 INFO:teuthology.orchestra.run.smithi186.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T09:26:19.716 INFO:teuthology.orchestra.run.smithi186.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T09:26:19.716 INFO:teuthology.orchestra.run.smithi186.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.716 INFO:teuthology.orchestra.run.smithi186.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T09:26:19.716 INFO:teuthology.orchestra.run.smithi186.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.716 INFO:teuthology.orchestra.run.smithi186.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T09:26:19.716 INFO:teuthology.orchestra.run.smithi186.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.717 INFO:teuthology.orchestra.run.smithi186.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.717 INFO:teuthology.orchestra.run.smithi186.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.717 INFO:teuthology.orchestra.run.smithi186.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T09:26:19.717 INFO:teuthology.orchestra.run.smithi186.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.717 INFO:teuthology.orchestra.run.smithi186.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.717 INFO:teuthology.orchestra.run.smithi186.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.717 INFO:teuthology.orchestra.run.smithi186.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T09:26:19.718 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:19.718 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Updated: 2019-12-09T09:26:19.718 INFO:teuthology.orchestra.run.smithi186.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.718 INFO:teuthology.orchestra.run.smithi186.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:19.718 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:19.718 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:26:19.849 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-test 2019-12-09T09:26:20.012 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:20.122 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:20.162 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:20.163 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:20.164 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:20.164 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:20.165 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:20.247 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:20.247 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:26:20.247 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:20.247 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:20.247 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository 2019-12-09T09:26:20.247 INFO:teuthology.orchestra.run.smithi033.stdout: Size 2019-12-09T09:26:20.248 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:20.248 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:26:20.248 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T09:26:20.248 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:26:20.248 INFO:teuthology.orchestra.run.smithi033.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T09:26:20.248 INFO:teuthology.orchestra.run.smithi033.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T09:26:20.248 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T09:26:20.249 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T09:26:20.249 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:20.249 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:26:20.249 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:20.249 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:26:20.249 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:20.249 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 38 M 2019-12-09T09:26:20.250 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 134 M 2019-12-09T09:26:20.250 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:26:20.833 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:21.184 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:26:21.184 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:21.185 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:21.202 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:21.232 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:21.233 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:21.235 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:21.235 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T09:26:21.235 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T09:26:21.236 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T09:26:21.238 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T09:26:21.241 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:21.241 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T09:26:21.604 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:21.828 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:21.829 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:26:21.829 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:21.829 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:21.829 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository 2019-12-09T09:26:21.830 INFO:teuthology.orchestra.run.smithi186.stdout: Size 2019-12-09T09:26:21.830 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:21.830 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:26:21.830 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T09:26:21.830 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:26:21.830 INFO:teuthology.orchestra.run.smithi186.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T09:26:21.830 INFO:teuthology.orchestra.run.smithi186.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T09:26:21.831 INFO:teuthology.orchestra.run.smithi186.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T09:26:21.831 INFO:teuthology.orchestra.run.smithi186.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T09:26:21.831 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:21.831 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:26:21.831 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:21.831 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:26:21.831 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:21.832 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 38 M 2019-12-09T09:26:21.832 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 134 M 2019-12-09T09:26:21.832 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:26:22.228 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:26:22.229 INFO:teuthology.orchestra.run.smithi033.stdout:Total 19 MB/s | 38 MB 00:01 2019-12-09T09:26:22.235 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:26:22.259 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:26:22.354 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:26:22.355 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:26:23.088 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T09:26:23.401 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T09:26:23.718 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T09:26:23.890 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:26:23.890 INFO:teuthology.orchestra.run.smithi186.stdout:Total 19 MB/s | 38 MB 00:02 2019-12-09T09:26:23.893 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:26:23.915 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:26:23.993 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:26:23.994 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:26:24.765 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T09:26:25.068 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T09:26:25.389 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T09:26:27.923 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:26:28.346 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:26:28.485 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:26:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T09:26:28.777 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T09:26:28.886 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:26:29.364 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T09:26:29.365 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:29.365 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:26:29.365 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:29.365 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:29.365 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:26:29.366 INFO:teuthology.orchestra.run.smithi033.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T09:26:29.366 INFO:teuthology.orchestra.run.smithi033.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T09:26:29.366 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:29.366 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:26:29.410 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:26:29.481 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph 2019-12-09T09:26:29.743 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:29.748 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:26:29.786 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:29.787 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:29.788 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:29.788 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:29.789 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:29.864 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:26:29.981 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T09:26:30.115 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T09:26:30.240 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T09:26:30.458 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:30.726 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T09:26:30.726 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:30.726 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:26:30.726 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:30.726 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:30.727 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:26:30.727 INFO:teuthology.orchestra.run.smithi186.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T09:26:30.727 INFO:teuthology.orchestra.run.smithi186.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T09:26:30.727 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:30.727 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:26:30.809 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:26:30.809 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:30.809 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:30.811 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.843 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.844 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.845 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.847 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:30.847 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:30.856 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph 2019-12-09T09:26:30.857 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:30.859 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.860 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.862 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.863 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.864 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.865 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:30.866 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:30.869 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:30.873 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:30.873 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T09:26:30.874 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T09:26:30.875 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T09:26:30.876 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:30.877 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:30.878 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:30.879 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:30.879 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:30.880 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T09:26:30.881 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T09:26:30.882 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T09:26:30.882 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T09:26:30.883 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T09:26:30.884 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:30.884 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T09:26:30.884 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:26:30.885 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:26:30.886 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T09:26:30.886 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T09:26:30.886 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T09:26:30.887 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T09:26:30.887 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T09:26:30.887 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T09:26:30.887 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:30.887 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T09:26:30.888 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T09:26:30.888 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T09:26:30.889 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:30.889 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T09:26:30.889 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T09:26:30.889 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:30.890 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T09:26:31.136 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:31.183 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:31.184 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:31.184 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:31.185 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:31.185 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:31.271 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:31.508 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:31.509 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:26:31.511 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:31.511 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository 2019-12-09T09:26:31.512 INFO:teuthology.orchestra.run.smithi033.stdout: Size 2019-12-09T09:26:31.512 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:31.512 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:26:31.515 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T09:26:31.515 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:26:31.516 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T09:26:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T09:26:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T09:26:31.517 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T09:26:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T09:26:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T09:26:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T09:26:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T09:26:31.518 INFO:teuthology.orchestra.run.smithi033.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T09:26:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T09:26:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T09:26:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T09:26:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T09:26:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T09:26:31.519 INFO:teuthology.orchestra.run.smithi033.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T09:26:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T09:26:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T09:26:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T09:26:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T09:26:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T09:26:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T09:26:31.520 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:31.521 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:26:31.521 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:31.521 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T09:26:31.521 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:31.523 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 28 M 2019-12-09T09:26:31.523 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 87 M 2019-12-09T09:26:31.523 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:26:31.847 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:32.195 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:26:32.195 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:32.195 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:32.197 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.228 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.229 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.230 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.231 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:32.231 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:32.241 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:32.243 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.244 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.246 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.247 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.248 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.249 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:26:32.250 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:32.252 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:32.257 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:32.257 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T09:26:32.258 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T09:26:32.259 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T09:26:32.259 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:32.260 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:32.266 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:32.267 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:32.268 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T09:26:32.268 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T09:26:32.269 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T09:26:32.269 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T09:26:32.269 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T09:26:32.269 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T09:26:32.269 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:32.269 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T09:26:32.270 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:26:32.270 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T09:26:32.278 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T09:26:32.278 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T09:26:32.278 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T09:26:32.279 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T09:26:32.279 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T09:26:32.279 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T09:26:32.279 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:32.279 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T09:26:32.279 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T09:26:32.279 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T09:26:32.280 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:32.280 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T09:26:32.280 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T09:26:32.280 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:32.280 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T09:26:32.641 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:32.871 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:32.871 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:26:32.876 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:32.876 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:32.876 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository 2019-12-09T09:26:32.877 INFO:teuthology.orchestra.run.smithi186.stdout: Size 2019-12-09T09:26:32.877 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:32.877 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:26:32.877 INFO:teuthology.orchestra.run.smithi186.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T09:26:32.877 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:26:32.877 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T09:26:32.877 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T09:26:32.878 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T09:26:32.878 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T09:26:32.878 INFO:teuthology.orchestra.run.smithi186.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T09:26:32.878 INFO:teuthology.orchestra.run.smithi186.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T09:26:32.878 INFO:teuthology.orchestra.run.smithi186.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T09:26:32.878 INFO:teuthology.orchestra.run.smithi186.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T09:26:32.878 INFO:teuthology.orchestra.run.smithi186.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T09:26:32.879 INFO:teuthology.orchestra.run.smithi186.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T09:26:32.879 INFO:teuthology.orchestra.run.smithi186.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T09:26:32.879 INFO:teuthology.orchestra.run.smithi186.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T09:26:32.879 INFO:teuthology.orchestra.run.smithi186.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T09:26:32.879 INFO:teuthology.orchestra.run.smithi186.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T09:26:32.879 INFO:teuthology.orchestra.run.smithi186.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T09:26:32.880 INFO:teuthology.orchestra.run.smithi186.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T09:26:32.880 INFO:teuthology.orchestra.run.smithi186.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T09:26:32.880 INFO:teuthology.orchestra.run.smithi186.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T09:26:32.880 INFO:teuthology.orchestra.run.smithi186.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T09:26:32.880 INFO:teuthology.orchestra.run.smithi186.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T09:26:32.880 INFO:teuthology.orchestra.run.smithi186.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T09:26:32.880 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:32.881 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:26:32.881 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:32.881 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T09:26:32.881 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:32.881 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 28 M 2019-12-09T09:26:32.881 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 87 M 2019-12-09T09:26:32.881 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:26:32.886 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:26:32.886 INFO:teuthology.orchestra.run.smithi033.stdout:Total 20 MB/s | 28 MB 00:01 2019-12-09T09:26:32.910 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:26:32.955 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:26:33.202 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:26:33.202 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:26:34.153 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T09:26:34.393 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:26:34.393 INFO:teuthology.orchestra.run.smithi186.stdout:Total 18 MB/s | 28 MB 00:01 2019-12-09T09:26:34.414 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:26:34.456 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:26:34.765 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:26:34.766 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:26:34.968 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T09:26:35.566 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T09:26:35.593 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T09:26:35.968 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T09:26:36.128 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T09:26:36.333 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T09:26:36.462 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T09:26:36.845 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T09:26:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T09:26:37.360 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T09:26:37.939 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T09:26:38.439 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T09:26:38.985 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T09:26:39.435 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T09:26:39.858 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T09:26:40.273 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T09:26:40.597 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T09:26:40.911 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T09:26:41.356 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T09:26:41.365 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T09:26:41.772 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T09:26:41.846 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:26:42.117 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T09:26:42.186 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T09:26:42.769 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T09:26:42.995 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T09:26:43.113 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T09:26:43.314 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T09:26:43.447 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:26:43.594 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T09:26:43.794 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T09:26:44.000 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T09:26:44.417 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T09:26:44.713 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T09:26:44.851 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T09:26:45.104 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T09:26:45.282 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T09:26:45.445 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T09:26:45.571 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T09:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T09:26:45.807 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T09:26:45.846 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T09:26:45.924 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T09:26:46.074 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T09:26:46.291 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T09:26:46.450 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T09:26:46.542 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T09:26:46.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T09:26:46.703 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T09:26:46.844 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T09:26:46.944 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T09:26:47.024 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T09:26:47.078 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T09:26:47.204 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T09:26:47.287 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:26:47.421 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T09:26:47.522 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T09:26:47.530 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T09:26:47.597 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T09:26:47.658 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T09:26:47.731 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T09:26:47.741 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T09:26:47.823 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T09:26:47.883 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T09:26:47.928 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T09:26:47.975 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T09:26:48.074 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T09:26:48.092 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T09:26:48.183 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T09:26:48.201 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T09:26:48.360 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T09:26:48.468 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T09:26:48.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T09:26:48.712 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T09:26:48.712 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:48.712 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:26:48.712 INFO:teuthology.orchestra.run.smithi033.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:48.712 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:48.713 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:26:48.713 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:48.713 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:48.713 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:48.713 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:48.713 INFO:teuthology.orchestra.run.smithi033.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T09:26:48.713 INFO:teuthology.orchestra.run.smithi033.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T09:26:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T09:26:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T09:26:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T09:26:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T09:26:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T09:26:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T09:26:48.714 INFO:teuthology.orchestra.run.smithi033.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T09:26:48.715 INFO:teuthology.orchestra.run.smithi033.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T09:26:48.716 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:48.716 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:26:48.753 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T09:26:48.834 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-daemon 2019-12-09T09:26:48.861 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T09:26:49.003 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T09:26:49.104 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T09:26:49.130 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:49.176 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:49.177 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:49.178 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:49.178 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:49.178 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:49.246 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T09:26:49.371 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T09:26:49.514 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T09:26:49.656 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T09:26:49.748 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T09:26:49.851 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:49.873 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T09:26:49.984 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T09:26:50.126 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T09:26:50.201 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:26:50.202 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:50.202 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:50.589 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:50.638 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T09:26:50.639 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:50.639 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:26:50.639 INFO:teuthology.orchestra.run.smithi186.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:50.639 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:50.639 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:26:50.639 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:50.639 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:50.640 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:50.640 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:50.640 INFO:teuthology.orchestra.run.smithi186.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T09:26:50.640 INFO:teuthology.orchestra.run.smithi186.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T09:26:50.640 INFO:teuthology.orchestra.run.smithi186.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T09:26:50.640 INFO:teuthology.orchestra.run.smithi186.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T09:26:50.640 INFO:teuthology.orchestra.run.smithi186.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T09:26:50.641 INFO:teuthology.orchestra.run.smithi186.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T09:26:50.641 INFO:teuthology.orchestra.run.smithi186.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T09:26:50.641 INFO:teuthology.orchestra.run.smithi186.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T09:26:50.641 INFO:teuthology.orchestra.run.smithi186.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T09:26:50.641 INFO:teuthology.orchestra.run.smithi186.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T09:26:50.641 INFO:teuthology.orchestra.run.smithi186.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T09:26:50.642 INFO:teuthology.orchestra.run.smithi186.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T09:26:50.642 INFO:teuthology.orchestra.run.smithi186.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T09:26:50.642 INFO:teuthology.orchestra.run.smithi186.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T09:26:50.642 INFO:teuthology.orchestra.run.smithi186.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T09:26:50.642 INFO:teuthology.orchestra.run.smithi186.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T09:26:50.642 INFO:teuthology.orchestra.run.smithi186.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T09:26:50.642 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:50.643 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:26:50.774 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-daemon 2019-12-09T09:26:50.819 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:50.819 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:26:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:50.820 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:50.820 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository 2019-12-09T09:26:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: Size 2019-12-09T09:26:50.821 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:50.821 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:26:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:26:50.821 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:50.821 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:26:50.821 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:50.822 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2019-12-09T09:26:50.822 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:50.822 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 17 k 2019-12-09T09:26:50.822 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 68 k 2019-12-09T09:26:50.822 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:26:51.039 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:51.083 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:51.084 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:51.085 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:51.085 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:51.085 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:51.197 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:26:51.205 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:26:51.207 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:26:51.207 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:26:51.735 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:26:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:52.078 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:52.102 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:26:52.447 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:52.608 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:26:52.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:52.608 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:26:52.608 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:52.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:52.609 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:26:52.683 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:52.683 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:26:52.684 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:52.684 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:52.684 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository 2019-12-09T09:26:52.684 INFO:teuthology.orchestra.run.smithi186.stdout: Size 2019-12-09T09:26:52.684 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:52.685 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:26:52.685 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:26:52.685 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:52.685 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:26:52.685 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:52.685 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2019-12-09T09:26:52.685 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:52.686 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 17 k 2019-12-09T09:26:52.686 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 68 k 2019-12-09T09:26:52.686 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:26:52.728 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr 2019-12-09T09:26:52.999 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:26:53.008 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:26:53.009 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:53.009 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:26:53.010 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:26:53.054 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:53.739 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:53.894 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:26:54.090 INFO:teuthology.orchestra.run.smithi033.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:26:54.090 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do 2019-12-09T09:26:54.152 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T09:26:54.329 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:54.366 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:54.367 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:54.367 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:54.368 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:54.368 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:54.428 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:26:54.428 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:54.428 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:26:54.428 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:26:54.429 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:54.429 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:26:54.548 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr 2019-12-09T09:26:54.818 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:54.862 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:54.863 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:54.863 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:54.864 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:54.864 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:54.997 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:55.339 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:26:55.339 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:55.339 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:55.342 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:26:55.374 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:26:55.375 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:55.375 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:55.375 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T09:26:55.375 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T09:26:55.384 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:26:55.385 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T09:26:55.529 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:55.761 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:55.870 INFO:teuthology.orchestra.run.smithi186.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:26:55.871 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do 2019-12-09T09:26:55.933 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T09:26:55.993 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:55.993 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:26:55.994 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:55.994 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:55.994 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:26:55.995 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:55.995 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:26:55.995 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T09:26:55.995 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:26:55.995 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T09:26:55.995 INFO:teuthology.orchestra.run.smithi033.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T09:26:55.996 INFO:teuthology.orchestra.run.smithi033.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T09:26:55.996 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:55.997 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:26:55.997 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:26:55.997 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T09:26:55.997 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:26:55.997 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 4.6 M 2019-12-09T09:26:55.997 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 20 M 2019-12-09T09:26:55.998 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:26:56.099 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:26:56.145 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:26:56.146 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:26:56.146 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:26:56.146 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:26:56.146 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:26:56.614 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:26:56.614 INFO:teuthology.orchestra.run.smithi033.stdout:Total 7.4 MB/s | 4.6 MB 00:00 2019-12-09T09:26:56.620 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:26:56.630 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:26:56.725 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:26:56.796 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:26:56.797 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:26:57.056 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:26:57.057 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:57.057 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:57.059 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:26:57.090 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:26:57.091 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:57.091 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:26:57.091 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T09:26:57.091 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T09:26:57.092 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:26:57.093 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T09:26:57.456 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:26:57.528 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T09:26:57.683 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:57.684 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:26:57.684 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:57.685 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:57.685 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2019-12-09T09:26:57.685 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:57.685 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:26:57.685 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T09:26:57.686 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:26:57.686 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T09:26:57.686 INFO:teuthology.orchestra.run.smithi186.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T09:26:57.686 INFO:teuthology.orchestra.run.smithi186.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T09:26:57.686 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:57.686 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:26:57.687 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:26:57.687 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T09:26:57.687 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:26:57.687 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 4.6 M 2019-12-09T09:26:57.687 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 20 M 2019-12-09T09:26:57.687 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:26:57.969 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T09:26:58.300 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:26:58.300 INFO:teuthology.orchestra.run.smithi186.stdout:Total 7.4 MB/s | 4.6 MB 00:00 2019-12-09T09:26:58.308 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:26:58.316 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:26:58.505 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:26:58.506 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:26:59.091 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T09:26:59.213 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T09:26:59.487 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T09:26:59.629 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T09:26:59.701 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T09:26:59.756 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T09:26:59.947 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T09:27:00.862 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T09:27:01.336 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T09:27:01.466 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T09:27:01.484 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T09:27:01.484 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:01.484 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:01.484 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:01.485 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:01.485 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:27:01.485 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:01.485 INFO:teuthology.orchestra.run.smithi033.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T09:27:01.485 INFO:teuthology.orchestra.run.smithi033.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T09:27:01.486 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:01.486 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:01.589 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T09:27:01.607 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T09:27:01.706 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T09:27:01.886 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:01.933 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:01.933 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:01.934 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:01.934 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:01.934 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:02.166 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T09:27:02.166 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:02.166 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:02.167 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:02.167 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:02.167 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:27:02.167 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:02.167 INFO:teuthology.orchestra.run.smithi186.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T09:27:02.167 INFO:teuthology.orchestra.run.smithi186.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T09:27:02.167 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:02.168 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:02.302 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T09:27:02.554 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:02.600 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:02.601 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:02.602 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:02.602 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:02.602 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:02.606 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:02.956 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:02.956 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:02.956 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:03.259 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:03.340 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:03.570 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:03.571 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:03.571 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:03.571 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:27:03.572 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:03.572 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T09:27:03.572 INFO:teuthology.orchestra.run.smithi033.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T09:27:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:03.573 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:03.573 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:03.573 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2019-12-09T09:27:03.573 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:03.573 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 82 k 2019-12-09T09:27:03.573 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 411 k 2019-12-09T09:27:03.574 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:03.601 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:03.601 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:03.602 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:03.931 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:03.934 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:03.969 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:03.989 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:03.990 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:04.198 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:04.198 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:04.200 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:04.200 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:04.200 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2019-12-09T09:27:04.200 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:04.201 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:04.201 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T09:27:04.201 INFO:teuthology.orchestra.run.smithi186.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T09:27:04.201 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:04.201 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:04.202 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:04.202 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2019-12-09T09:27:04.202 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:04.202 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 82 k 2019-12-09T09:27:04.202 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 411 k 2019-12-09T09:27:04.203 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:04.494 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:04.497 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:04.542 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:04.542 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:04.672 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:27:05.103 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:27:05.103 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:05.103 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:05.104 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:05.104 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:05.104 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:05.225 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T09:27:05.289 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:27:05.467 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:05.510 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:05.511 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:05.511 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:05.511 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:05.512 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:05.829 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T09:27:05.829 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:05.829 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:05.829 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:05.830 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:05.830 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:05.963 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T09:27:06.174 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:06.219 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:06.263 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:06.264 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:06.265 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:06.265 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:06.265 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:06.518 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:06.519 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:06.519 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:06.521 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:27:06.552 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:06.553 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T09:27:06.560 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:27:06.561 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:27:06.563 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:27:06.563 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:06.563 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T09:27:06.564 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T09:27:06.565 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:27:06.567 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:27:06.567 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:06.567 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T09:27:06.932 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:06.950 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:07.181 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:07.181 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:07.182 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:27:07.182 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:07.182 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:07.182 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:07.183 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:07.184 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:07.184 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:27:07.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:07.185 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 19 M 2019-12-09T09:27:07.185 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 104 M 2019-12-09T09:27:07.185 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:07.273 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:07.273 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:07.273 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:07.275 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:27:07.306 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:07.306 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T09:27:07.313 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:27:07.314 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:27:07.316 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T09:27:07.316 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:07.316 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T09:27:07.317 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T09:27:07.318 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:27:07.320 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T09:27:07.320 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:07.320 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T09:27:07.687 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:07.914 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:07.914 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:07.915 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:07.915 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:07.916 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2019-12-09T09:27:07.916 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:07.916 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:07.916 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local 2019-12-09T09:27:07.916 INFO:teuthology.orchestra.run.smithi186.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T09:27:07.916 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:27:07.917 INFO:teuthology.orchestra.run.smithi186.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T09:27:07.917 INFO:teuthology.orchestra.run.smithi186.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T09:27:07.917 INFO:teuthology.orchestra.run.smithi186.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T09:27:07.917 INFO:teuthology.orchestra.run.smithi186.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T09:27:07.917 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:07.917 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:07.917 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:07.918 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:27:07.918 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:07.918 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 19 M 2019-12-09T09:27:07.918 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 104 M 2019-12-09T09:27:07.918 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:08.005 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:08.006 INFO:teuthology.orchestra.run.smithi033.stdout:Total 23 MB/s | 19 MB 00:00 2019-12-09T09:27:08.020 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:08.055 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:08.236 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:08.237 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:08.591 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:08.591 INFO:teuthology.orchestra.run.smithi186.stdout:Total 28 MB/s | 19 MB 00:00 2019-12-09T09:27:08.603 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:08.631 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:08.782 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:08.782 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:09.123 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T09:27:09.613 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T09:27:09.637 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T09:27:10.169 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T09:27:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T09:27:11.973 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T09:27:13.145 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T09:27:13.473 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T09:27:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T09:27:13.677 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T09:27:13.726 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T09:27:13.800 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T09:27:13.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T09:27:14.107 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T09:27:14.244 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T09:27:14.368 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T09:27:14.406 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T09:27:14.406 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:14.406 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:14.406 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:14.406 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:14.406 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:27:14.407 INFO:teuthology.orchestra.run.smithi033.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T09:27:14.407 INFO:teuthology.orchestra.run.smithi033.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T09:27:14.407 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:14.407 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:14.502 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T09:27:14.542 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-rook 2019-12-09T09:27:14.578 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T09:27:14.808 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:14.854 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:14.855 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:14.856 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:14.856 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:14.856 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:15.038 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi186.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi186.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:15.039 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:15.165 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-rook 2019-12-09T09:27:15.426 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:15.471 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:15.471 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:15.472 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:15.472 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:15.473 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:15.518 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:15.873 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:15.873 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:15.874 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:15.875 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:27:15.905 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:15.905 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T09:27:15.907 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:15.908 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:15.909 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:15.910 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T09:27:15.912 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T09:27:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T09:27:15.913 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:27:15.914 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:27:15.914 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:27:15.915 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T09:27:15.916 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T09:27:15.917 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:15.917 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T09:27:15.917 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T09:27:15.920 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T09:27:15.920 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T09:27:15.921 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T09:27:15.921 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:15.921 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T09:27:16.130 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:16.302 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:16.470 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:16.470 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:16.471 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:16.472 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:27:16.503 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:16.504 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T09:27:16.505 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:16.506 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:16.507 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:16.508 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T09:27:16.509 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:16.509 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T09:27:16.509 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T09:27:16.510 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T09:27:16.510 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:27:16.511 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:27:16.512 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T09:27:16.513 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T09:27:16.513 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T09:27:16.514 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:16.514 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T09:27:16.514 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T09:27:16.515 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T09:27:16.516 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T09:27:16.517 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T09:27:16.517 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:16.517 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T09:27:16.536 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:16.536 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:16.537 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:16.537 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:16.537 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:27:16.537 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:16.537 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:16.537 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T09:27:16.537 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:27:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T09:27:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T09:27:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T09:27:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T09:27:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T09:27:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T09:27:16.538 INFO:teuthology.orchestra.run.smithi033.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T09:27:16.539 INFO:teuthology.orchestra.run.smithi033.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T09:27:16.539 INFO:teuthology.orchestra.run.smithi033.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T09:27:16.539 INFO:teuthology.orchestra.run.smithi033.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T09:27:16.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:16.540 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:16.540 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:16.540 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T09:27:16.540 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:16.541 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 1.6 M 2019-12-09T09:27:16.541 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 24 M 2019-12-09T09:27:16.541 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:16.879 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:17.108 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:17.109 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:17.112 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:17.112 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:17.112 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T09:27:17.113 INFO:teuthology.orchestra.run.smithi186.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T09:27:17.114 INFO:teuthology.orchestra.run.smithi186.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T09:27:17.114 INFO:teuthology.orchestra.run.smithi186.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T09:27:17.114 INFO:teuthology.orchestra.run.smithi186.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T09:27:17.114 INFO:teuthology.orchestra.run.smithi186.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T09:27:17.114 INFO:teuthology.orchestra.run.smithi186.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T09:27:17.114 INFO:teuthology.orchestra.run.smithi186.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T09:27:17.114 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:17.115 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:17.115 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:17.118 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T09:27:17.119 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:17.119 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 1.6 M 2019-12-09T09:27:17.119 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 24 M 2019-12-09T09:27:17.119 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:17.340 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:17.340 INFO:teuthology.orchestra.run.smithi033.stdout:Total 2.1 MB/s | 1.6 MB 00:00 2019-12-09T09:27:17.354 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:17.365 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:17.431 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:17.432 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:17.898 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:17.899 INFO:teuthology.orchestra.run.smithi186.stdout:Total 2.1 MB/s | 1.6 MB 00:00 2019-12-09T09:27:17.908 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:17.917 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:17.977 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:17.977 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:18.188 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T09:27:18.559 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T09:27:18.771 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T09:27:18.917 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T09:27:19.213 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T09:27:19.240 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T09:27:19.586 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T09:27:19.648 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T09:27:19.975 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T09:27:20.013 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T09:27:20.310 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T09:27:20.410 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T09:27:20.607 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T09:27:20.784 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T09:27:21.225 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T09:27:21.599 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T09:27:21.963 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T09:27:22.373 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T09:27:22.659 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T09:27:22.758 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T09:27:22.866 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T09:27:22.947 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T09:27:22.992 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T09:27:23.084 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T09:27:23.193 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T09:27:23.352 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T09:27:23.388 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T09:27:23.469 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T09:27:23.620 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T09:27:23.720 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T09:27:23.732 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T09:27:23.821 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T09:27:23.882 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T09:27:24.007 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T09:27:24.157 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T09:27:24.249 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T09:27:24.367 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T09:27:24.381 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T09:27:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:24.382 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:24.382 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:27:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T09:27:24.382 INFO:teuthology.orchestra.run.smithi033.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T09:27:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T09:27:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T09:27:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T09:27:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T09:27:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T09:27:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T09:27:24.383 INFO:teuthology.orchestra.run.smithi033.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T09:27:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T09:27:24.384 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:24.384 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:24.484 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T09:27:24.502 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-ssh 2019-12-09T09:27:24.609 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T09:27:24.701 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T09:27:24.767 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:24.802 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T09:27:24.813 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:24.814 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:24.815 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:24.886 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T09:27:25.380 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T09:27:25.380 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:25.381 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:25.381 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:25.381 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:25.381 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:27:25.381 INFO:teuthology.orchestra.run.smithi186.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T09:27:25.381 INFO:teuthology.orchestra.run.smithi186.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T09:27:25.381 INFO:teuthology.orchestra.run.smithi186.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T09:27:25.382 INFO:teuthology.orchestra.run.smithi186.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T09:27:25.382 INFO:teuthology.orchestra.run.smithi186.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T09:27:25.382 INFO:teuthology.orchestra.run.smithi186.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T09:27:25.382 INFO:teuthology.orchestra.run.smithi186.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T09:27:25.382 INFO:teuthology.orchestra.run.smithi186.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T09:27:25.382 INFO:teuthology.orchestra.run.smithi186.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T09:27:25.382 INFO:teuthology.orchestra.run.smithi186.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T09:27:25.383 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:25.383 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:25.492 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:25.536 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-mgr-ssh 2019-12-09T09:27:25.800 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:25.843 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:25.843 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:25.844 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:25.844 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:25.845 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:25.846 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:25.846 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:25.847 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:27:25.847 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:25.878 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:25.878 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T09:27:25.879 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T09:27:25.879 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:25.879 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T09:27:26.266 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:26.500 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:26.501 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:26.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:26.502 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:26.502 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:27:26.502 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:26.502 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:26.502 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T09:27:26.503 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:27:26.503 INFO:teuthology.orchestra.run.smithi033.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T09:27:26.503 INFO:teuthology.orchestra.run.smithi033.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T09:27:26.503 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:26.503 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:26.503 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:26.503 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T09:27:26.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:26.504 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 316 k 2019-12-09T09:27:26.504 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 1.1 M 2019-12-09T09:27:26.504 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:26.505 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:26.847 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:26.847 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:26.847 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:26.849 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T09:27:26.880 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:26.880 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T09:27:26.881 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T09:27:26.881 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:26.882 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T09:27:27.040 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:27.040 INFO:teuthology.orchestra.run.smithi033.stdout:Total 588 kB/s | 316 kB 00:00 2019-12-09T09:27:27.044 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:27.053 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:27.134 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:27.134 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:27.247 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:27.476 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:27.476 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:27.477 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:27.477 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:27.477 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2019-12-09T09:27:27.477 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:27.477 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:27.478 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T09:27:27.478 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:27:27.478 INFO:teuthology.orchestra.run.smithi186.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T09:27:27.478 INFO:teuthology.orchestra.run.smithi186.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T09:27:27.478 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:27.479 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:27.479 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:27.479 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T09:27:27.479 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:27.479 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 316 k 2019-12-09T09:27:27.479 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 1.1 M 2019-12-09T09:27:27.480 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:27.910 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T09:27:27.989 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:27.989 INFO:teuthology.orchestra.run.smithi186.stdout:Total 618 kB/s | 316 kB 00:00 2019-12-09T09:27:27.993 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:28.001 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:28.073 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:28.074 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:28.197 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T09:27:28.500 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:27:28.620 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T09:27:28.745 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T09:27:28.968 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T09:27:29.270 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T09:27:29.289 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:27:29.289 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T09:27:29.290 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:29.291 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:29.418 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-fuse 2019-12-09T09:27:29.588 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:27:29.679 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:29.692 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T09:27:29.724 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:29.726 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:29.726 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:29.726 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:29.726 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:29.793 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T09:27:30.287 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T09:27:30.287 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:30.288 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:30.288 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:30.288 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:30.288 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:27:30.288 INFO:teuthology.orchestra.run.smithi186.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T09:27:30.288 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:30.289 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:30.386 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:30.419 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install ceph-fuse 2019-12-09T09:27:30.676 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:30.721 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:30.721 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:30.722 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:30.722 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:30.722 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:30.737 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:30.737 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:30.738 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:30.749 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:27:30.783 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:27:30.783 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:30.783 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:27:30.784 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:30.786 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:30.786 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:30.788 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:30.788 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:30.788 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:27:30.795 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T09:27:30.795 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T09:27:31.180 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:31.384 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:31.413 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:31.414 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:31.415 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository 2019-12-09T09:27:31.415 INFO:teuthology.orchestra.run.smithi033.stdout: Size 2019-12-09T09:27:31.416 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:31.416 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T09:27:31.416 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:27:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T09:27:31.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T09:27:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T09:27:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T09:27:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:31.417 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:27:31.417 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 10 M 2019-12-09T09:27:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 49 M 2019-12-09T09:27:31.418 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:31.707 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:31.708 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:31.708 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:31.718 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:27:31.752 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:27:31.752 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:31.752 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:27:31.753 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:31.754 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:31.755 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:31.756 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T09:27:31.757 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:31.757 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T09:27:31.762 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T09:27:31.763 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T09:27:31.804 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:31.804 INFO:teuthology.orchestra.run.smithi033.stdout:Total 26 MB/s | 10 MB 00:00 2019-12-09T09:27:31.821 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:31.841 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:31.940 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:31.941 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:32.127 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:32.354 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:32.354 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:32.355 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:32.355 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:32.355 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository 2019-12-09T09:27:32.356 INFO:teuthology.orchestra.run.smithi186.stdout: Size 2019-12-09T09:27:32.356 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:32.356 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:32.356 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T09:27:32.356 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:27:32.356 INFO:teuthology.orchestra.run.smithi186.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T09:27:32.357 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:32.358 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 10 M 2019-12-09T09:27:32.358 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 49 M 2019-12-09T09:27:32.358 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:32.774 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:32.774 INFO:teuthology.orchestra.run.smithi186.stdout:Total 24 MB/s | 10 MB 00:00 2019-12-09T09:27:32.787 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:32.805 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:32.980 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:32.980 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:34.297 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T09:27:34.826 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T09:27:35.395 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T09:27:35.769 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T09:27:35.980 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T09:27:36.430 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T09:27:36.686 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:27:36.825 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:27:36.860 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T09:27:36.947 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T09:27:37.093 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T09:27:37.219 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T09:27:37.380 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T09:27:37.711 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T09:27:37.731 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T09:27:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:37.732 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:37.732 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:27:37.732 INFO:teuthology.orchestra.run.smithi033.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T09:27:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T09:27:37.733 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:37.733 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:37.871 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install libcephfs2 2019-12-09T09:27:38.115 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:38.131 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T09:27:38.157 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:38.158 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:38.159 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:38.159 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:38.159 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:38.239 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T09:27:38.381 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T09:27:38.474 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T09:27:38.833 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:38.968 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T09:27:38.968 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:38.968 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:38.968 INFO:teuthology.orchestra.run.smithi186.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:38.969 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:38.969 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:27:38.969 INFO:teuthology.orchestra.run.smithi186.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T09:27:38.969 INFO:teuthology.orchestra.run.smithi186.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T09:27:38.969 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:38.970 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:39.103 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install libcephfs2 2019-12-09T09:27:39.178 INFO:teuthology.orchestra.run.smithi033.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:27:39.178 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do 2019-12-09T09:27:39.240 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install libcephfs-devel 2019-12-09T09:27:39.360 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:39.405 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:39.406 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:39.406 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:39.406 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:39.407 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:39.413 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:39.450 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:39.451 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:39.452 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:39.452 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:39.452 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:40.067 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:40.076 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:40.406 INFO:teuthology.orchestra.run.smithi186.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:27:40.407 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do 2019-12-09T09:27:40.417 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:40.418 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:40.418 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:40.418 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:27:40.450 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:40.450 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:40.468 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install libcephfs-devel 2019-12-09T09:27:40.628 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:40.663 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:40.664 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:40.664 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:40.664 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:40.665 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:40.849 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:41.080 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:41.081 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:41.081 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:41.082 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository 2019-12-09T09:27:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: Size 2019-12-09T09:27:41.082 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:41.082 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:41.082 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:27:41.082 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:27:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T09:27:41.083 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:41.083 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:41.083 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:41.084 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T09:27:41.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:41.084 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 170 k 2019-12-09T09:27:41.084 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 679 k 2019-12-09T09:27:41.084 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:41.249 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:41.430 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:41.430 INFO:teuthology.orchestra.run.smithi033.stdout:Total 487 kB/s | 170 kB 00:00 2019-12-09T09:27:41.433 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:41.444 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:41.450 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:41.450 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:41.577 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:41.578 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:41.578 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:41.579 INFO:teuthology.orchestra.run.smithi186.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T09:27:41.610 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:41.610 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:41.984 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:42.025 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:27:42.214 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:42.214 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:42.215 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:42.215 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:42.215 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository 2019-12-09T09:27:42.215 INFO:teuthology.orchestra.run.smithi186.stdout: Size 2019-12-09T09:27:42.216 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:42.216 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:42.216 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T09:27:42.216 INFO:teuthology.orchestra.run.smithi186.stdout:Installing for dependencies: 2019-12-09T09:27:42.216 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T09:27:42.216 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:42.216 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:42.217 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:42.217 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T09:27:42.217 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:42.217 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 170 k 2019-12-09T09:27:42.217 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 679 k 2019-12-09T09:27:42.217 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:42.306 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:27:42.426 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:27:42.592 INFO:teuthology.orchestra.run.smithi186.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:27:42.592 INFO:teuthology.orchestra.run.smithi186.stdout:Total 453 kB/s | 170 kB 00:00 2019-12-09T09:27:42.595 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:42.602 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:42.603 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:42.603 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:42.910 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:27:42.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:42.911 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:42.911 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:42.911 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:42.911 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:27:42.912 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:42.912 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:42.912 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:43.066 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install librados2 2019-12-09T09:27:43.331 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:43.377 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:43.378 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:43.379 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:43.379 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:43.379 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:43.806 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:27:44.054 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:44.101 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:27:44.223 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T09:27:44.407 INFO:teuthology.orchestra.run.smithi033.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:27:44.407 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do 2019-12-09T09:27:44.469 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install librbd1 2019-12-09T09:27:44.639 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:44.666 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T09:27:44.667 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:44.667 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:44.667 INFO:teuthology.orchestra.run.smithi186.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:44.667 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:44.667 INFO:teuthology.orchestra.run.smithi186.stdout:Dependency Installed: 2019-12-09T09:27:44.667 INFO:teuthology.orchestra.run.smithi186.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:44.667 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:44.668 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:44.676 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:44.677 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:44.677 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:44.677 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:44.678 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:44.804 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install librados2 2019-12-09T09:27:45.035 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:45.077 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:45.077 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:45.078 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:45.078 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:45.079 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:45.300 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:45.640 INFO:teuthology.orchestra.run.smithi033.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:27:45.640 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do 2019-12-09T09:27:45.702 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install python-ceph 2019-12-09T09:27:45.723 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:45.918 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:45.960 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:45.960 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:45.961 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:45.962 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:46.075 INFO:teuthology.orchestra.run.smithi186.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:27:46.075 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do 2019-12-09T09:27:46.136 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install librbd1 2019-12-09T09:27:46.354 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:46.388 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:46.389 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:46.389 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:46.390 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:46.390 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:46.640 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:46.979 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:46.980 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:46.980 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:47.048 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:47.370 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:47.381 INFO:teuthology.orchestra.run.smithi186.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T09:27:47.382 INFO:teuthology.orchestra.run.smithi186.stdout:Nothing to do 2019-12-09T09:27:47.444 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install python-ceph 2019-12-09T09:27:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:47.564 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:47.565 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository 2019-12-09T09:27:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: Size 2019-12-09T09:27:47.565 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:47.565 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:47.565 INFO:teuthology.orchestra.run.smithi033.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T09:27:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:47.566 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:47.566 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:47.566 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2019-12-09T09:27:47.566 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:47.566 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 2.2 k 2019-12-09T09:27:47.566 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 0 2019-12-09T09:27:47.567 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:47.658 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:47.702 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:47.702 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:47.703 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:47.703 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:47.704 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:47.854 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:47.856 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:47.858 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:47.858 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:48.351 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:48.389 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:48.688 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:48.689 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:48.689 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:48.874 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:48.874 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:48.874 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:48.874 INFO:teuthology.orchestra.run.smithi033.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:48.874 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:48.875 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:48.993 INFO:teuthology.orchestra.run.smithi033:> sudo yum -y install rbd-fuse 2019-12-09T09:27:49.061 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:49.241 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:49.248 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:49.249 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:49.249 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:49.249 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:49.250 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository 2019-12-09T09:27:49.250 INFO:teuthology.orchestra.run.smithi186.stdout: Size 2019-12-09T09:27:49.251 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:49.251 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:49.251 INFO:teuthology.orchestra.run.smithi186.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T09:27:49.251 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:49.252 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:49.252 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:49.252 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2019-12-09T09:27:49.253 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:49.253 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 2.2 k 2019-12-09T09:27:49.253 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 0 2019-12-09T09:27:49.253 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:49.287 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:49.287 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:49.287 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:49.287 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:49.288 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:49.517 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:49.525 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:49.525 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:49.525 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:49.956 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:50.102 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:50.302 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:27:50.302 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:27:50.303 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:50.627 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:50.628 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:50.628 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:50.628 INFO:teuthology.orchestra.run.smithi186.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:50.628 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:50.628 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:50.702 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:50.748 INFO:teuthology.orchestra.run.smithi186:> sudo yum -y install rbd-fuse 2019-12-09T09:27:50.894 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:50.894 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:27:50.895 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:50.895 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:50.895 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:27:50.895 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:50.895 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:27:50.895 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T09:27:50.896 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:27:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:27:50.896 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2019-12-09T09:27:50.896 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 76 k 2019-12-09T09:27:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 200 k 2019-12-09T09:27:50.897 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:27:51.015 INFO:teuthology.orchestra.run.smithi186.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:27:51.061 INFO:teuthology.orchestra.run.smithi186.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:27:51.062 INFO:teuthology.orchestra.run.smithi186.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:27:51.063 INFO:teuthology.orchestra.run.smithi186.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:27:51.063 INFO:teuthology.orchestra.run.smithi186.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:27:51.063 INFO:teuthology.orchestra.run.smithi186.stdout: * updates: mirror.cogentco.com 2019-12-09T09:27:51.237 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:27:51.259 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:27:51.261 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:27:51.261 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:27:51.719 INFO:teuthology.orchestra.run.smithi186.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:27:51.952 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:Resolving Dependencies 2019-12-09T09:27:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:--> Running transaction check 2019-12-09T09:27:52.063 INFO:teuthology.orchestra.run.smithi186.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T09:27:52.370 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:52.370 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:52.370 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:27:52.370 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:52.370 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:27:52.370 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:27:52.441 INFO:teuthology.orchestra.run.smithi186.stdout:--> Finished Dependency Resolution 2019-12-09T09:27:52.482 DEBUG:teuthology.parallel:result is None 2019-12-09T09:27:52.627 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:52.628 INFO:teuthology.orchestra.run.smithi186.stdout:Dependencies Resolved 2019-12-09T09:27:52.628 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:52.628 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:52.629 INFO:teuthology.orchestra.run.smithi186.stdout: Package Arch Version Repository Size 2019-12-09T09:27:52.629 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:52.629 INFO:teuthology.orchestra.run.smithi186.stdout:Installing: 2019-12-09T09:27:52.629 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T09:27:52.629 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:52.629 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction Summary 2019-12-09T09:27:52.630 INFO:teuthology.orchestra.run.smithi186.stdout:================================================================================ 2019-12-09T09:27:52.630 INFO:teuthology.orchestra.run.smithi186.stdout:Install 1 Package 2019-12-09T09:27:52.630 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:52.630 INFO:teuthology.orchestra.run.smithi186.stdout:Total download size: 76 k 2019-12-09T09:27:52.630 INFO:teuthology.orchestra.run.smithi186.stdout:Installed size: 200 k 2019-12-09T09:27:52.630 INFO:teuthology.orchestra.run.smithi186.stdout:Downloading packages: 2019-12-09T09:27:52.963 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction check 2019-12-09T09:27:52.986 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction test 2019-12-09T09:27:52.988 INFO:teuthology.orchestra.run.smithi186.stdout:Transaction test succeeded 2019-12-09T09:27:52.988 INFO:teuthology.orchestra.run.smithi186.stdout:Running transaction 2019-12-09T09:27:53.842 INFO:teuthology.orchestra.run.smithi186.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:54.367 INFO:teuthology.orchestra.run.smithi186.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T09:27:54.367 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:54.367 INFO:teuthology.orchestra.run.smithi186.stdout:Installed: 2019-12-09T09:27:54.367 INFO:teuthology.orchestra.run.smithi186.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:54.367 INFO:teuthology.orchestra.run.smithi186.stdout: 2019-12-09T09:27:54.367 INFO:teuthology.orchestra.run.smithi186.stdout:Complete! 2019-12-09T09:27:54.488 DEBUG:teuthology.parallel:result is None 2019-12-09T09:27:54.488 INFO:teuthology.orchestra.run.smithi033:> sudo lsb_release -is 2019-12-09T09:27:54.643 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS 2019-12-09T09:27:54.644 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:27:54.645 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:27:54.795 INFO:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T09:27:54.929 INFO:teuthology.orchestra.run.smithi033.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:54.929 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T09:27:54.930 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T09:27:54.930 INFO:teuthology.orchestra.run.smithi186:> sudo lsb_release -is 2019-12-09T09:27:55.079 INFO:teuthology.orchestra.run.smithi186.stdout:CentOS 2019-12-09T09:27:55.081 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T09:27:55.082 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:27:55.264 INFO:teuthology.orchestra.run.smithi186:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T09:27:55.388 INFO:teuthology.orchestra.run.smithi186.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T09:27:55.388 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T09:27:55.388 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T09:27:55.389 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-12-09T09:27:55.389 INFO:teuthology.orchestra.run.smithi033:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T09:27:55.513 INFO:teuthology.orchestra.run.smithi186:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T09:27:55.635 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-12-09T09:27:55.635 INFO:teuthology.orchestra.run.smithi033:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T09:27:55.757 INFO:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T09:27:55.870 INFO:teuthology.orchestra.run.smithi186:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T09:27:55.995 INFO:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T09:27:56.096 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-12-09T09:27:56.097 INFO:teuthology.orchestra.run.smithi033:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T09:27:56.215 INFO:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T09:27:56.331 INFO:teuthology.orchestra.run.smithi186:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T09:27:56.453 INFO:teuthology.orchestra.run.smithi186:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T09:27:56.543 INFO:teuthology.run_tasks:Running task ceph... 2019-12-09T09:27:56.626 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2019-12-09T09:27:56.627 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 777 /var/log/ceph 2019-12-09T09:27:56.632 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 777 /var/log/ceph 2019-12-09T09:27:56.748 INFO:tasks.ceph:Disabling ceph logrotate... 2019-12-09T09:27:56.749 INFO:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T09:27:56.754 INFO:teuthology.orchestra.run.smithi186:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T09:27:56.867 INFO:tasks.ceph:Creating extra log directories... 2019-12-09T09:27:56.867 INFO:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T09:27:56.871 INFO:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T09:27:56.984 INFO:tasks.ceph:Creating ceph cluster ceph... 2019-12-09T09:27:56.985 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 1, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'osd shutdown pgref assert': True, 'debug journal': 20, 'bluestore allocator': 'bitmap', 'mon osd nearfull ratio': 0.8, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'osd fast shutdown': False, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}} 2019-12-09T09:27:56.985 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh']}}}, {'ragweed': {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}}}, {'s3tests-java': {'client.0': {'force-repo': 'https://github.com/ceph/java_s3tests.git', 'force-branch': 'master'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master'}}}, {'swift': {'client.0': {'rgw_server': 'client.0'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.13052', 'owner': 'scheduled_ofriedma@teuthology', 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'targets': {'smithi033.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOi70/1lLZEdHzIrJ9izBFnq+89y3RD78C2AkrMEqStAtUlu1ZyS34ityJlJ7OzmUTbcl/I52cbRPvHdIeQjgbRBcMo2EF5pM+3+iW+H7vdrAdVGAVLE2pzUG9cnCKCmD7jTtYUJ2NJDCkVU9Mhz59ZvkT7ZWz33fVVhy2JQD477B5h3PyNqgELosKAdxRYkViX4aIh95ibVLoq0ipkBppbf494GelyI9JmpXHfV4EtrFr0+vKah6BFb3dzxJu5e1igDGHg16udXkqwKXrs3TQJkiZeI2+UpgJR3TayL6v2J4hF5TGysDlTNNJEEcyt5fnIr6CNmEr2ziq10Bm1jHT', 'smithi186.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDY0LwYYJs/HJCINnWA3ZlN0URRZ57HW3F2VRemQKoDvkYDvSj4fJznS8yJYxdUUJGL5Ow0kMvZqYg3mg1A+xSMYY5ivzY6th6AUpb9vxl4IwNNN0Tbsg3vE72VyyIqKU2OPK/cuwDkj6zxIJk6VKSfLTOCQnkUALCNcQBE5R4i4g8njGnscjpE3/IxN9awxjzhmIl2Tw4rDmuTnHNH6sbkdwg57Hct4OBl9WHTlITLbrYaCUB6LYYYZROAKvC7Y8WYFyzDBebaVLcHXnwBI74RlntKDPSDHREWR+7ZvKS69+G6COzUAfFZ3W81YiVqWlkhlAZM3QOL9ehMFpiCjLlZ'}, 'job_id': '4584223', 'tube': 'smithi', 'suite_branch': 'wip-ofriedma-dynamic-disable', 'priority': 60, 'first_in_suite': False, 'branch': 'wip-ofriedma-dynamic-disable', 'suite': 'rgw', 'email': 'ofriedma@redhat.com', 'archive_path': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584223', 'description': 'rgw/verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml}', 'nuke-on-error': True, 'repo': 'https://github.com/ceph/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'name': 'ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'os_type': 'centos', 'overrides': {'s3tests': {'slow_backend': True}, 'ceph-deploy': {'fs': 'xfs', 'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'mon osd full ratio': 0.9, 'mon osd backfillfull_ratio': 0.85, 'bluestore fsck on mount': True, 'mon osd nearfull ratio': 0.8, 'debug bluestore': 20, 'debug bluefs': 20, 'osd objectstore': 'bluestore', 'bluestore block size': 96636764160, 'debug rocksdb': 10, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95, 'bdev async discard': True}}}, 'rgw': {'frontend': 'civetweb', 'compression type': 'random', 'storage classes': 'LUKEWARM, FROZEN', 'ec-data-pool': True}, 'workunit': {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'branch': 'wip-ofriedma-dynamic-disable'}, 'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 1, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'osd shutdown pgref assert': True, 'debug journal': 20, 'bluestore allocator': 'bitmap', 'mon osd nearfull ratio': 0.8, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'osd fast shutdown': False, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'fs': 'xfs', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, 'admin_socket': {'branch': 'wip-ofriedma-dynamic-disable'}, 'thrashosds': {'bdev_inject_crash_probability': 0.5, 'bdev_inject_crash': 2}}} 2019-12-09T09:27:56.985 INFO:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T09:27:56.989 INFO:teuthology.orchestra.run.smithi186:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T09:27:57.068 INFO:teuthology.orchestra.run.smithi033:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T09:27:57.072 INFO:teuthology.orchestra.run.smithi186:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T09:27:57.271 DEBUG:teuthology.orchestra.remote:smithi033:/scratch_devs is 72B 2019-12-09T09:27:57.362 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:27:57.363 INFO:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T09:27:57.442 INFO:teuthology.orchestra.run.smithi033.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T09:27:57.442 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:57.443 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 103207 Links: 1 2019-12-09T09:27:57.443 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:57.443 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:57.443 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2019-12-09 09:26:45.170815871 +0000 2019-12-09T09:27:57.443 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2019-12-09 09:24:57.269738981 +0000 2019-12-09T09:27:57.443 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2019-12-09 09:24:57.269738981 +0000 2019-12-09T09:27:57.444 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2019-12-09T09:27:57.474 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2019-12-09T09:27:57.474 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2019-12-09T09:27:57.474 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes (512 B) copied, 0.00045644 s, 1.1 MB/s 2019-12-09T09:27:57.479 INFO:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T09:27:57.609 INFO:teuthology.orchestra.run.smithi033.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T09:27:57.609 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:57.609 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 103247 Links: 1 2019-12-09T09:27:57.609 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:57.609 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:57.610 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2019-12-09 09:26:45.170815871 +0000 2019-12-09T09:27:57.610 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2019-12-09 09:24:57.784730578 +0000 2019-12-09T09:27:57.610 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2019-12-09 09:24:57.784730578 +0000 2019-12-09T09:27:57.610 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2019-12-09T09:27:57.640 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2019-12-09T09:27:57.640 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2019-12-09T09:27:57.640 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes (512 B) copied, 0.00045586 s, 1.1 MB/s 2019-12-09T09:27:57.649 INFO:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T09:27:57.727 INFO:teuthology.orchestra.run.smithi033.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T09:27:57.727 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:57.728 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 106902 Links: 1 2019-12-09T09:27:57.728 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:57.728 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:57.728 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2019-12-09 09:26:45.171815871 +0000 2019-12-09T09:27:57.729 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2019-12-09 09:24:58.294722259 +0000 2019-12-09T09:27:57.729 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2019-12-09 09:24:58.294722259 +0000 2019-12-09T09:27:57.729 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2019-12-09T09:27:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2019-12-09T09:27:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2019-12-09T09:27:57.754 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes (512 B) copied, 0.000448295 s, 1.1 MB/s 2019-12-09T09:27:57.758 INFO:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T09:27:57.841 INFO:teuthology.orchestra.run.smithi033.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T09:27:57.841 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:57.841 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 106949 Links: 1 2019-12-09T09:27:57.841 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:57.841 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:57.842 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2019-12-09 09:26:45.171815871 +0000 2019-12-09T09:27:57.854 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2019-12-09 09:24:58.788714199 +0000 2019-12-09T09:27:57.854 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2019-12-09 09:24:58.788714199 +0000 2019-12-09T09:27:57.854 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2019-12-09T09:27:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2019-12-09T09:27:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2019-12-09T09:27:57.872 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes (512 B) copied, 0.000411772 s, 1.2 MB/s 2019-12-09T09:27:57.877 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T09:27:57.877 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:27:57.878 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T09:27:57.878 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:27:57.973 DEBUG:teuthology.orchestra.remote:smithi186:/scratch_devs is 72B 2019-12-09T09:27:58.053 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:27:58.053 INFO:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T09:27:58.139 INFO:teuthology.orchestra.run.smithi186.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T09:27:58.139 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:58.140 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 107982 Links: 1 2019-12-09T09:27:58.140 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:58.140 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:58.140 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2019-12-09 09:26:46.874317244 +0000 2019-12-09T09:27:58.140 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2019-12-09 09:24:53.396209924 +0000 2019-12-09T09:27:58.140 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2019-12-09 09:24:53.396209924 +0000 2019-12-09T09:27:58.140 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2019-12-09T09:27:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2019-12-09T09:27:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2019-12-09T09:27:58.169 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes (512 B) copied, 0.000456682 s, 1.1 MB/s 2019-12-09T09:27:58.175 INFO:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T09:27:58.252 INFO:teuthology.orchestra.run.smithi186.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T09:27:58.253 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:58.253 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 102852 Links: 1 2019-12-09T09:27:58.253 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:58.253 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:58.253 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2019-12-09 09:26:46.874317244 +0000 2019-12-09T09:27:58.253 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2019-12-09 09:24:53.886203077 +0000 2019-12-09T09:27:58.253 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2019-12-09 09:24:53.886203077 +0000 2019-12-09T09:27:58.254 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2019-12-09T09:27:58.281 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2019-12-09T09:27:58.281 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2019-12-09T09:27:58.281 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes (512 B) copied, 0.000394339 s, 1.3 MB/s 2019-12-09T09:27:58.286 INFO:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T09:27:58.357 INFO:teuthology.orchestra.run.smithi186.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T09:27:58.358 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:58.358 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 83924 Links: 1 2019-12-09T09:27:58.358 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:58.358 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:58.358 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2019-12-09 09:26:46.874317244 +0000 2019-12-09T09:27:58.358 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2019-12-09 09:24:54.336196788 +0000 2019-12-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2019-12-09 09:24:54.336196788 +0000 2019-12-09T09:27:58.359 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2019-12-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2019-12-09T09:27:58.383 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2019-12-09T09:27:58.384 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes (512 B) copied, 0.000352372 s, 1.5 MB/s 2019-12-09T09:27:58.388 INFO:teuthology.orchestra.run.smithi186:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T09:27:58.450 INFO:teuthology.orchestra.run.smithi186.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T09:27:58.450 INFO:teuthology.orchestra.run.smithi186.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T09:27:58.450 INFO:teuthology.orchestra.run.smithi186.stdout:Device: 5h/5d Inode: 110672 Links: 1 2019-12-09T09:27:58.450 INFO:teuthology.orchestra.run.smithi186.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T09:27:58.450 INFO:teuthology.orchestra.run.smithi186.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T09:27:58.451 INFO:teuthology.orchestra.run.smithi186.stdout:Access: 2019-12-09 09:26:46.875317244 +0000 2019-12-09T09:27:58.451 INFO:teuthology.orchestra.run.smithi186.stdout:Modify: 2019-12-09 09:24:54.803190262 +0000 2019-12-09T09:27:58.451 INFO:teuthology.orchestra.run.smithi186.stdout:Change: 2019-12-09 09:24:54.803190262 +0000 2019-12-09T09:27:58.451 INFO:teuthology.orchestra.run.smithi186.stdout: Birth: - 2019-12-09T09:27:58.470 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records in 2019-12-09T09:27:58.470 INFO:teuthology.orchestra.run.smithi186.stderr:1+0 records out 2019-12-09T09:27:58.471 INFO:teuthology.orchestra.run.smithi186.stderr:512 bytes (512 B) copied, 0.000331033 s, 1.5 MB/s 2019-12-09T09:27:58.474 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T09:27:58.474 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T09:27:58.474 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T09:27:58.474 INFO:tasks.ceph:dev map: {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:27:58.474 INFO:tasks.ceph:Generating config... 2019-12-09T09:27:58.476 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2019-12-09T09:27:58.476 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2019-12-09T09:27:58.476 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2019-12-09T09:27:58.476 INFO:tasks.ceph:[global] mon client directed command retry = 5 2019-12-09T09:27:58.476 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2019-12-09T09:27:58.476 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2019-12-09T09:27:58.476 INFO:tasks.ceph:[osd] debug ms = 1 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] debug osd = 25 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] debug bluestore = 20 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] debug bluefs = 20 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] debug rocksdb = 10 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2019-12-09T09:27:58.477 INFO:tasks.ceph:[osd] debug journal = 20 2019-12-09T09:27:58.478 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2019-12-09T09:27:58.478 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2019-12-09T09:27:58.478 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2019-12-09T09:27:58.480 INFO:tasks.ceph:[osd] bdev async discard = True 2019-12-09T09:27:58.480 INFO:tasks.ceph:[osd] osd fast shutdown = False 2019-12-09T09:27:58.480 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2019-12-09T09:27:58.480 INFO:tasks.ceph:[osd] debug filestore = 20 2019-12-09T09:27:58.481 INFO:tasks.ceph:[osd] bdev enable discard = True 2019-12-09T09:27:58.481 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2019-12-09T09:27:58.481 INFO:tasks.ceph:[mon] debug mon = 20 2019-12-09T09:27:58.481 INFO:tasks.ceph:[mon] debug paxos = 20 2019-12-09T09:27:58.481 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2019-12-09T09:27:58.481 INFO:tasks.ceph:[mon] debug ms = 1 2019-12-09T09:27:58.481 INFO:tasks.ceph:[client] debug rgw = 20 2019-12-09T09:27:58.481 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2019-12-09T09:27:58.482 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2019-12-09T09:27:58.482 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2019-12-09T09:27:58.482 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2019-12-09T09:27:58.482 INFO:tasks.ceph:Setting up mon.a... 2019-12-09T09:27:58.482 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2019-12-09T09:27:58.669 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.keyring 2019-12-09T09:27:58.674 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2019-12-09T09:27:58.819 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:27:58.922 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.33'), ('mon.c', '[v2:172.21.15.33:3301,v1:172.21.15.33:6790]'), ('mon.b', '172.21.15.186')] 2019-12-09T09:27:58.922 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.33 --addv c '[v2:172.21.15.33:3301,v1:172.21.15.33:6790]' --add b 172.21.15.186 --print /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:27:59.061 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:27:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: generated fsid a3644d3a-627a-4518-94ad-53a356f98d9f 2019-12-09T09:27:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 0 2019-12-09T09:27:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:fsid a3644d3a-627a-4518-94ad-53a356f98d9f 2019-12-09T09:27:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:last_changed 2019-12-09T09:27:59.062428+0000 2019-12-09T09:27:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:created 2019-12-09T09:27:59.062428+0000 2019-12-09T09:27:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:min_mon_release 0 (unknown) 2019-12-09T09:27:59.062 INFO:teuthology.orchestra.run.smithi033.stdout:0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2019-12-09T09:27:59.063 INFO:teuthology.orchestra.run.smithi033.stdout:1: [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] mon.b 2019-12-09T09:27:59.063 INFO:teuthology.orchestra.run.smithi033.stdout:2: [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] mon.c 2019-12-09T09:27:59.063 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2019-12-09T09:27:59.064 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a3644d3a-627a-4518-94ad-53a356f98d9f... 2019-12-09T09:27:59.065 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T09:27:59.069 INFO:teuthology.orchestra.run.smithi186:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T09:27:59.290 INFO:tasks.ceph:Creating admin key on mon.a... 2019-12-09T09:27:59.291 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2019-12-09T09:27:59.420 INFO:tasks.ceph:Copying monmap to all nodes... 2019-12-09T09:27:59.486 DEBUG:teuthology.orchestra.remote:smithi033:/etc/ceph/ceph.keyring is 206B 2019-12-09T09:27:59.669 DEBUG:teuthology.orchestra.remote:smithi033:/home/ubuntu/cephtest/ceph.monmap is 379B 2019-12-09T09:27:59.766 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:27:59.767 INFO:teuthology.orchestra.run.smithi033:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:27:59.913 INFO:teuthology.orchestra.run.smithi033:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:27:59.989 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:27:59.990 INFO:teuthology.orchestra.run.smithi186:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T09:28:00.134 INFO:teuthology.orchestra.run.smithi186:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:00.202 INFO:tasks.ceph:Setting up mon nodes... 2019-12-09T09:28:00.202 INFO:tasks.ceph:Setting up mgr nodes... 2019-12-09T09:28:00.202 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T09:28:00.382 INFO:teuthology.orchestra.run.smithi033.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T09:28:00.387 INFO:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T09:28:00.607 INFO:teuthology.orchestra.run.smithi186.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T09:28:00.612 INFO:tasks.ceph:Setting up mds nodes... 2019-12-09T09:28:00.612 INFO:tasks.ceph_client:Setting up client nodes... 2019-12-09T09:28:00.612 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2019-12-09T09:28:00.763 INFO:teuthology.orchestra.run.smithi033.stdout:creating /etc/ceph/ceph.client.0.keyring 2019-12-09T09:28:00.796 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2019-12-09T09:28:00.940 INFO:teuthology.orchestra.run.smithi186.stdout:creating /etc/ceph/ceph.client.1.keyring 2019-12-09T09:28:00.970 INFO:tasks.ceph:Running mkfs on osd nodes... 2019-12-09T09:28:00.970 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi033.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi186.front.sepia.ceph.com'): {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'}} 2019-12-09T09:28:00.970 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2019-12-09T09:28:01.089 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:01.090 INFO:tasks.ceph:{} 2019-12-09T09:28:01.090 INFO:tasks.ceph:osd.0 2019-12-09T09:28:01.090 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:28:01.090 INFO:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T09:28:01.676 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:01.676 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:01.676 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:01.676 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:01.676 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:01.677 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:01.677 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:01.677 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:01.677 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:01.679 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2019-12-09T09:28:01.679 INFO:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2019-12-09T09:28:01.846 INFO:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2019-12-09T09:28:01.960 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2019-12-09T09:28:02.061 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:02.061 INFO:tasks.ceph:{} 2019-12-09T09:28:02.061 INFO:tasks.ceph:osd.1 2019-12-09T09:28:02.061 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:28:02.062 INFO:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T09:28:02.641 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:02.641 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:02.641 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:02.642 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:02.642 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:02.642 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:02.642 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:02.642 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:02.644 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:02.644 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2019-12-09T09:28:02.645 INFO:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2019-12-09T09:28:02.799 INFO:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2019-12-09T09:28:02.909 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2019-12-09T09:28:02.998 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:02.998 INFO:tasks.ceph:{} 2019-12-09T09:28:02.998 INFO:tasks.ceph:osd.2 2019-12-09T09:28:02.998 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:28:02.998 INFO:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T09:28:03.576 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:03.576 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:03.577 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:03.577 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:03.577 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:03.577 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:03.577 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:03.579 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2019-12-09T09:28:03.579 INFO:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2019-12-09T09:28:03.752 INFO:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2019-12-09T09:28:03.854 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2019-12-09T09:28:03.961 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:03.961 INFO:tasks.ceph:{} 2019-12-09T09:28:03.961 INFO:tasks.ceph:osd.3 2019-12-09T09:28:03.962 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:28:03.962 INFO:teuthology.orchestra.run.smithi033:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T09:28:04.523 INFO:teuthology.orchestra.run.smithi033.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:04.524 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:04.524 INFO:teuthology.orchestra.run.smithi033.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:04.524 INFO:teuthology.orchestra.run.smithi033.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:04.524 INFO:teuthology.orchestra.run.smithi033.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:04.524 INFO:teuthology.orchestra.run.smithi033.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:04.524 INFO:teuthology.orchestra.run.smithi033.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:04.525 INFO:teuthology.orchestra.run.smithi033.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:04.525 INFO:teuthology.orchestra.run.smithi033.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:04.526 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi033.front.sepia.ceph.com -o noatime 2019-12-09T09:28:04.526 INFO:teuthology.orchestra.run.smithi033:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2019-12-09T09:28:04.678 INFO:teuthology.orchestra.run.smithi033:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2019-12-09T09:28:04.779 INFO:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:04.916 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:04.912+0000 7f1cf5eadb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2019-12-09T09:28:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:04.913+0000 7f1cf5eadb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2019-12-09T09:28:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:04.913+0000 7f1cf5eadb40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-09T09:28:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:04.913+0000 7f1cf5eadb40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-09T09:28:04.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:04.913+0000 7f1cf5eadb40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-09T09:28:04.921 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:04.920+0000 7f1cf5eadb40 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2019-12-09T09:28:07.230 INFO:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2019-12-09T09:28:07.346 INFO:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:07.492 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:07.491+0000 7fa90cf95b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2019-12-09T09:28:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:07.492+0000 7fa90cf95b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2019-12-09T09:28:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:07.492+0000 7fa90cf95b40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-09T09:28:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:07.492+0000 7fa90cf95b40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-09T09:28:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:07.492+0000 7fa90cf95b40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-09T09:28:07.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:07.493+0000 7fa90cf95b40 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2019-12-09T09:28:09.801 INFO:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2019-12-09T09:28:09.918 INFO:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:10.074+0000 7fb69dfbeb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2019-12-09T09:28:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:10.075+0000 7fb69dfbeb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2019-12-09T09:28:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:10.075+0000 7fb69dfbeb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-09T09:28:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:10.075+0000 7fb69dfbeb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-09T09:28:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:10.075+0000 7fb69dfbeb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-09T09:28:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:10.076+0000 7fb69dfbeb40 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2019-12-09T09:28:12.385 INFO:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2019-12-09T09:28:12.504 INFO:teuthology.orchestra.run.smithi033:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:12.664+0000 7f61e566eb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2019-12-09T09:28:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:12.665+0000 7f61e566eb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2019-12-09T09:28:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:12.665+0000 7f61e566eb40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-09T09:28:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:12.665+0000 7f61e566eb40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-09T09:28:12.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:12.665+0000 7f61e566eb40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-09T09:28:12.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:28:12.667+0000 7f61e566eb40 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2019-12-09T09:28:14.974 INFO:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2019-12-09T09:28:15.098 INFO:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2019-12-09T09:28:15.222 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:15.229 INFO:tasks.ceph:{} 2019-12-09T09:28:15.229 INFO:tasks.ceph:osd.4 2019-12-09T09:28:15.229 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:28:15.230 INFO:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T09:28:15.824 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:15.824 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:15.824 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:15.824 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:15.824 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:15.825 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:15.825 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:15.825 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:15.825 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:15.826 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2019-12-09T09:28:15.827 INFO:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4 2019-12-09T09:28:15.995 INFO:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2019-12-09T09:28:16.110 INFO:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2019-12-09T09:28:16.196 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:16.197 INFO:tasks.ceph:{} 2019-12-09T09:28:16.197 INFO:tasks.ceph:osd.5 2019-12-09T09:28:16.197 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:28:16.197 INFO:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T09:28:16.738 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:16.738 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:16.739 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:16.739 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:16.739 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:16.739 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:16.739 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:16.739 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:16.739 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:16.741 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2019-12-09T09:28:16.741 INFO:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-5 2019-12-09T09:28:16.881 INFO:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2019-12-09T09:28:16.982 INFO:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2019-12-09T09:28:17.064 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:17.065 INFO:tasks.ceph:{} 2019-12-09T09:28:17.065 INFO:tasks.ceph:osd.6 2019-12-09T09:28:17.065 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:28:17.065 INFO:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T09:28:17.633 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:17.634 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:17.634 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:17.634 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:17.635 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:17.635 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:17.635 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:17.636 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:17.636 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:17.636 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2019-12-09T09:28:17.637 INFO:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2019-12-09T09:28:17.816 INFO:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2019-12-09T09:28:17.936 INFO:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2019-12-09T09:28:18.037 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T09:28:18.038 INFO:tasks.ceph:{} 2019-12-09T09:28:18.038 INFO:tasks.ceph:osd.7 2019-12-09T09:28:18.038 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:28:18.038 INFO:teuthology.orchestra.run.smithi186:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T09:28:18.603 INFO:teuthology.orchestra.run.smithi186.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T09:28:18.604 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T09:28:18.604 INFO:teuthology.orchestra.run.smithi186.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T09:28:18.604 INFO:teuthology.orchestra.run.smithi186.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T09:28:18.604 INFO:teuthology.orchestra.run.smithi186.stdout: = sunit=0 swidth=0 blks 2019-12-09T09:28:18.604 INFO:teuthology.orchestra.run.smithi186.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T09:28:18.604 INFO:teuthology.orchestra.run.smithi186.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T09:28:18.605 INFO:teuthology.orchestra.run.smithi186.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T09:28:18.605 INFO:teuthology.orchestra.run.smithi186.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T09:28:18.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi186.front.sepia.ceph.com -o noatime 2019-12-09T09:28:18.607 INFO:teuthology.orchestra.run.smithi186:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-7 2019-12-09T09:28:18.736 INFO:teuthology.orchestra.run.smithi186:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2019-12-09T09:28:18.829 INFO:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:18.945 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:18.945+0000 7f7336f34b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2019-12-09T09:28:18.950 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:18.950+0000 7f7336f34b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2019-12-09T09:28:18.950 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:18.950+0000 7f7336f34b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-09T09:28:18.950 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:18.950+0000 7f7336f34b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-09T09:28:18.950 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:18.950+0000 7f7336f34b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-09T09:28:18.953 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:18.954+0000 7f7336f34b40 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2019-12-09T09:28:21.261 INFO:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2019-12-09T09:28:21.374 INFO:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:21.509 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:21.509+0000 7f6c0ee4db40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2019-12-09T09:28:21.510 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:21.510+0000 7f6c0ee4db40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2019-12-09T09:28:21.510 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:21.510+0000 7f6c0ee4db40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-09T09:28:21.510 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:21.510+0000 7f6c0ee4db40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-09T09:28:21.510 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:21.510+0000 7f6c0ee4db40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-09T09:28:21.510 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:21.511+0000 7f6c0ee4db40 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2019-12-09T09:28:23.818 INFO:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2019-12-09T09:28:23.931 INFO:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:24.079 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:24.079+0000 7f8d07d76b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2019-12-09T09:28:24.080 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:24.079+0000 7f8d07d76b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2019-12-09T09:28:24.080 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:24.079+0000 7f8d07d76b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-09T09:28:24.080 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:24.079+0000 7f8d07d76b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-09T09:28:24.080 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:24.079+0000 7f8d07d76b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-09T09:28:24.081 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:24.081+0000 7f8d07d76b40 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2019-12-09T09:28:26.390 INFO:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2019-12-09T09:28:26.509 INFO:teuthology.orchestra.run.smithi186:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:26.641 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:26.641+0000 7f2b76b06b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2019-12-09T09:28:26.642 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:26.642+0000 7f2b76b06b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2019-12-09T09:28:26.642 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:26.642+0000 7f2b76b06b40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-09T09:28:26.643 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:26.642+0000 7f2b76b06b40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-09T09:28:26.643 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:26.642+0000 7f2b76b06b40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-09T09:28:26.643 INFO:teuthology.orchestra.run.smithi186.stderr:2019-12-09T09:28:26.643+0000 7f2b76b06b40 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2019-12-09T09:28:28.952 INFO:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2019-12-09T09:28:29.062 INFO:tasks.ceph:Reading keys from all nodes... 2019-12-09T09:28:29.063 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:29.157 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.VGvnqokvTe 2019-12-09T09:28:29.158 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.VGvnqokvTe 2019-12-09T09:28:29.268 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.VGvnqokvTe 2019-12-09T09:28:29.476 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.VGvnqokvTe is 56B 2019-12-09T09:28:29.559 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.VGvnqokvTe 2019-12-09T09:28:29.626 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:29.694 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.KKw9xzfbtD 2019-12-09T09:28:29.695 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.KKw9xzfbtD 2019-12-09T09:28:29.790 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.KKw9xzfbtD 2019-12-09T09:28:29.980 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.KKw9xzfbtD is 56B 2019-12-09T09:28:30.069 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.KKw9xzfbtD 2019-12-09T09:28:30.157 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:30.243 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.NcU8hEboIY 2019-12-09T09:28:30.244 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.NcU8hEboIY 2019-12-09T09:28:30.358 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.NcU8hEboIY 2019-12-09T09:28:30.570 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.NcU8hEboIY is 56B 2019-12-09T09:28:30.661 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.NcU8hEboIY 2019-12-09T09:28:30.740 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:30.870 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.mgdFSD6nFv 2019-12-09T09:28:30.870 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.mgdFSD6nFv 2019-12-09T09:28:30.974 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.mgdFSD6nFv 2019-12-09T09:28:31.152 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.mgdFSD6nFv is 56B 2019-12-09T09:28:31.223 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.mgdFSD6nFv 2019-12-09T09:28:31.289 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:31.366 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.DA1CvHjW50 2019-12-09T09:28:31.367 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.DA1CvHjW50 2019-12-09T09:28:31.467 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.DA1CvHjW50 2019-12-09T09:28:31.628 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.DA1CvHjW50 is 56B 2019-12-09T09:28:31.698 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.DA1CvHjW50 2019-12-09T09:28:31.777 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:31.867 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.kZCRQqV43I 2019-12-09T09:28:31.868 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.kZCRQqV43I 2019-12-09T09:28:31.966 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.kZCRQqV43I 2019-12-09T09:28:32.135 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.kZCRQqV43I is 56B 2019-12-09T09:28:32.221 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.kZCRQqV43I 2019-12-09T09:28:32.303 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:32.366 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.FdzANGIv6J 2019-12-09T09:28:32.366 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.FdzANGIv6J 2019-12-09T09:28:32.457 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.FdzANGIv6J 2019-12-09T09:28:32.623 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.FdzANGIv6J is 56B 2019-12-09T09:28:32.690 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.FdzANGIv6J 2019-12-09T09:28:32.753 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:32.875 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.kJtkkARvP2 2019-12-09T09:28:32.876 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.kJtkkARvP2 2019-12-09T09:28:32.973 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.kJtkkARvP2 2019-12-09T09:28:33.134 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.kJtkkARvP2 is 56B 2019-12-09T09:28:33.198 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.kJtkkARvP2 2019-12-09T09:28:33.256 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:33.379 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.DUcoztoNwC 2019-12-09T09:28:33.380 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.DUcoztoNwC 2019-12-09T09:28:33.491 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.DUcoztoNwC 2019-12-09T09:28:33.692 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.DUcoztoNwC is 56B 2019-12-09T09:28:33.783 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.DUcoztoNwC 2019-12-09T09:28:33.868 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:33.998 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.pj4w62U2eo 2019-12-09T09:28:33.998 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.pj4w62U2eo 2019-12-09T09:28:34.105 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.pj4w62U2eo 2019-12-09T09:28:34.282 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.pj4w62U2eo is 56B 2019-12-09T09:28:34.352 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.pj4w62U2eo 2019-12-09T09:28:34.513 DEBUG:teuthology.orchestra.remote:smithi033:/etc/ceph/ceph.client.0.keyring is 59B 2019-12-09T09:28:34.702 DEBUG:teuthology.orchestra.remote:smithi186:/etc/ceph/ceph.client.1.keyring is 59B 2019-12-09T09:28:34.795 INFO:tasks.ceph:Adding keys to all mons... 2019-12-09T09:28:34.796 INFO:teuthology.orchestra.run.smithi033:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T09:28:34.799 INFO:teuthology.orchestra.run.smithi186:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T09:28:34.905 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.y] 2019-12-09T09:28:34.905 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCgE+5dFtvdFhAACyydEkkQsxx9b7wms8BLXw== 2019-12-09T09:28:34.905 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.0] 2019-12-09T09:28:34.905 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCkE+5d/6B7NhAA5RA584qf1TyxzgHP58ZsUg== 2019-12-09T09:28:34.905 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.1] 2019-12-09T09:28:34.906 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCnE+5d8IdqHRAA56U9y3W8rJ5IqI5lAITGjg== 2019-12-09T09:28:34.906 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.2] 2019-12-09T09:28:34.906 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCqE+5dMM+LBBAA8wZ0bPb2pEa3vUiAT53q8Q== 2019-12-09T09:28:34.906 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.3] 2019-12-09T09:28:34.906 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCsE+5dhq69JxAAuj19nbsdGfpjNCrvrYuoGQ== 2019-12-09T09:28:34.906 INFO:teuthology.orchestra.run.smithi186.stdout:[mgr.x] 2019-12-09T09:28:34.906 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCgE+5dEwlEJBAAkAnndax36AS3E42hsWL9WQ== 2019-12-09T09:28:34.907 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.4] 2019-12-09T09:28:34.907 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCyE+5dP61yOBAAVieUMLHTx8yh7tjYOWQEdg== 2019-12-09T09:28:34.907 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.5] 2019-12-09T09:28:34.907 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQC1E+5d9e5uHhAAelNOu3F66ugBhrSLcpIO/A== 2019-12-09T09:28:34.907 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.6] 2019-12-09T09:28:34.907 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQC4E+5d2lvJBBAAugmhASIDuI9uLim5FMEdTw== 2019-12-09T09:28:34.907 INFO:teuthology.orchestra.run.smithi186.stdout:[osd.7] 2019-12-09T09:28:34.908 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQC6E+5dWNBRJhAAU+xNVkqRryY2Prar+D9TDQ== 2019-12-09T09:28:34.908 INFO:teuthology.orchestra.run.smithi186.stdout:[client.0] 2019-12-09T09:28:34.908 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCgE+5dLXSSLRAAAgjvZKJC+iaByOKpMPH3Iw== 2019-12-09T09:28:34.908 INFO:teuthology.orchestra.run.smithi186.stdout:[client.1] 2019-12-09T09:28:34.908 INFO:teuthology.orchestra.run.smithi186.stdout: key = AQCgE+5dpxsjOBAAsiJnwqmNaL1Egwt1KPXxZA== 2019-12-09T09:28:34.916 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.y] 2019-12-09T09:28:34.917 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCgE+5dFtvdFhAACyydEkkQsxx9b7wms8BLXw== 2019-12-09T09:28:34.917 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.0] 2019-12-09T09:28:34.917 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCkE+5d/6B7NhAA5RA584qf1TyxzgHP58ZsUg== 2019-12-09T09:28:34.917 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.1] 2019-12-09T09:28:34.917 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCnE+5d8IdqHRAA56U9y3W8rJ5IqI5lAITGjg== 2019-12-09T09:28:34.917 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.2] 2019-12-09T09:28:34.918 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCqE+5dMM+LBBAA8wZ0bPb2pEa3vUiAT53q8Q== 2019-12-09T09:28:34.918 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.3] 2019-12-09T09:28:34.918 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCsE+5dhq69JxAAuj19nbsdGfpjNCrvrYuoGQ== 2019-12-09T09:28:34.918 INFO:teuthology.orchestra.run.smithi033.stdout:[mgr.x] 2019-12-09T09:28:34.918 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCgE+5dEwlEJBAAkAnndax36AS3E42hsWL9WQ== 2019-12-09T09:28:34.918 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.4] 2019-12-09T09:28:34.918 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCyE+5dP61yOBAAVieUMLHTx8yh7tjYOWQEdg== 2019-12-09T09:28:34.919 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.5] 2019-12-09T09:28:34.919 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC1E+5d9e5uHhAAelNOu3F66ugBhrSLcpIO/A== 2019-12-09T09:28:34.919 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.6] 2019-12-09T09:28:34.919 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC4E+5d2lvJBBAAugmhASIDuI9uLim5FMEdTw== 2019-12-09T09:28:34.919 INFO:teuthology.orchestra.run.smithi033.stdout:[osd.7] 2019-12-09T09:28:34.919 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQC6E+5dWNBRJhAAU+xNVkqRryY2Prar+D9TDQ== 2019-12-09T09:28:34.919 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2019-12-09T09:28:34.920 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCgE+5dLXSSLRAAAgjvZKJC+iaByOKpMPH3Iw== 2019-12-09T09:28:34.920 INFO:teuthology.orchestra.run.smithi033.stdout:[client.1] 2019-12-09T09:28:34.920 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCgE+5dpxsjOBAAsiJnwqmNaL1Egwt1KPXxZA== 2019-12-09T09:28:34.921 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:28:34.928 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:28:35.086 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.090 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.258 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.268 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.448 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.452 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.613 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.617 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.780 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:28:35.784 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T09:28:35.954 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:35.961 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:36.129 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:36.134 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:36.302 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:36.306 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:36.481 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:36.484 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T09:28:36.644 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:28:36.648 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:28:36.825 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:28:36.830 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T09:28:37.005 INFO:tasks.ceph:Running mkfs on mon nodes... 2019-12-09T09:28:37.006 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2019-12-09T09:28:37.125 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:28:37.310 INFO:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2019-12-09T09:28:37.420 INFO:teuthology.orchestra.run.smithi033:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2019-12-09T09:28:37.526 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:28:37.716 INFO:teuthology.orchestra.run.smithi033:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2019-12-09T09:28:37.837 INFO:teuthology.orchestra.run.smithi186:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2019-12-09T09:28:37.955 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T09:28:38.132 INFO:teuthology.orchestra.run.smithi186:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2019-12-09T09:28:38.237 INFO:teuthology.orchestra.run.smithi033:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:38.241 INFO:teuthology.orchestra.run.smithi186:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T09:28:38.327 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2019-12-09T09:28:38.328 DEBUG:teuthology.misc:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:28:38.328 INFO:tasks.ceph.mon.a:Restarting daemon 2019-12-09T09:28:38.328 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2019-12-09T09:28:38.332 INFO:tasks.ceph.mon.a:Started 2019-12-09T09:28:38.332 DEBUG:teuthology.misc:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:28:38.332 INFO:tasks.ceph.mon.c:Restarting daemon 2019-12-09T09:28:38.332 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2019-12-09T09:28:38.335 INFO:tasks.ceph.mon.c:Started 2019-12-09T09:28:38.335 DEBUG:teuthology.misc:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T09:28:38.335 INFO:tasks.ceph.mon.b:Restarting daemon 2019-12-09T09:28:38.336 INFO:teuthology.orchestra.run.smithi186:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2019-12-09T09:28:38.339 INFO:tasks.ceph.mon.b:Started 2019-12-09T09:28:38.339 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2019-12-09T09:28:38.340 INFO:tasks.ceph.mgr.y:Restarting daemon 2019-12-09T09:28:38.340 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2019-12-09T09:28:38.343 INFO:tasks.ceph.mgr.y:Started 2019-12-09T09:28:38.343 INFO:tasks.ceph.mgr.x:Restarting daemon 2019-12-09T09:28:38.343 INFO:teuthology.orchestra.run.smithi186:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2019-12-09T09:28:38.346 INFO:tasks.ceph.mgr.x:Started 2019-12-09T09:28:38.346 INFO:tasks.ceph:Setting crush tunables to default 2019-12-09T09:28:38.346 INFO:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd crush tunables default 2019-12-09T09:28:42.428 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:00:03.892 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:42.453 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:00:03.938 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:42.622 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:00:04.086 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:28:55.134 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2019-12-09T09:28:55.147 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2019-12-09T09:28:55.147 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:55.199 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.ClmYlh1pRe 2019-12-09T09:28:55.241 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.ClmYlh1pRe 2019-12-09T09:28:55.314 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.ClmYlh1pRe 2019-12-09T09:28:55.433 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.ClmYlh1pRe is 37B 2019-12-09T09:28:55.481 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.ClmYlh1pRe 2019-12-09T09:28:55.525 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:55.622 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.M6l6qupf5C 2019-12-09T09:28:55.622 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.M6l6qupf5C 2019-12-09T09:28:55.685 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.M6l6qupf5C 2019-12-09T09:28:55.796 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.M6l6qupf5C is 37B 2019-12-09T09:28:55.846 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.M6l6qupf5C 2019-12-09T09:28:55.890 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:55.974 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.zQhgo6pXRc 2019-12-09T09:28:55.975 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.zQhgo6pXRc 2019-12-09T09:28:56.040 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.zQhgo6pXRc 2019-12-09T09:28:56.134 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.zQhgo6pXRc is 37B 2019-12-09T09:28:56.180 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.zQhgo6pXRc 2019-12-09T09:28:56.219 INFO:teuthology.orchestra.run.smithi033:> mktemp 2019-12-09T09:28:56.321 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.K0TOhizERb 2019-12-09T09:28:56.321 INFO:teuthology.orchestra.run.smithi033:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.K0TOhizERb 2019-12-09T09:28:56.395 INFO:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.K0TOhizERb 2019-12-09T09:28:56.491 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.K0TOhizERb is 37B 2019-12-09T09:28:56.535 INFO:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.K0TOhizERb 2019-12-09T09:28:56.575 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:56.634 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.MQxwxhgwU3 2019-12-09T09:28:56.635 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.MQxwxhgwU3 2019-12-09T09:28:56.722 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.MQxwxhgwU3 2019-12-09T09:28:56.827 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.MQxwxhgwU3 is 37B 2019-12-09T09:28:56.878 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.MQxwxhgwU3 2019-12-09T09:28:56.922 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:57.035 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.VnzyDt05Z0 2019-12-09T09:28:57.036 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.VnzyDt05Z0 2019-12-09T09:28:57.124 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.VnzyDt05Z0 2019-12-09T09:28:57.249 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.VnzyDt05Z0 is 37B 2019-12-09T09:28:57.304 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.VnzyDt05Z0 2019-12-09T09:28:57.352 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:57.445 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.kPIGf9mmYz 2019-12-09T09:28:57.446 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.kPIGf9mmYz 2019-12-09T09:28:57.510 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.kPIGf9mmYz 2019-12-09T09:28:57.614 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.kPIGf9mmYz is 37B 2019-12-09T09:28:57.673 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.kPIGf9mmYz 2019-12-09T09:28:57.725 INFO:teuthology.orchestra.run.smithi186:> mktemp 2019-12-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi186.stdout:/tmp/tmp.3vDdHbxZfl 2019-12-09T09:28:57.772 INFO:teuthology.orchestra.run.smithi186:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.3vDdHbxZfl 2019-12-09T09:28:57.835 INFO:teuthology.orchestra.run.smithi186:> sudo chmod 0666 /tmp/tmp.3vDdHbxZfl 2019-12-09T09:28:57.946 DEBUG:teuthology.orchestra.remote:smithi186:/tmp/tmp.3vDdHbxZfl is 37B 2019-12-09T09:28:57.998 INFO:teuthology.orchestra.run.smithi186:> rm -fr /tmp/tmp.3vDdHbxZfl 2019-12-09T09:28:58.043 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 6f8f10c0-106f-478a-a436-88ac562a6b42 0 2019-12-09T09:28:59.064 INFO:teuthology.orchestra.run.smithi186.stdout:0 2019-12-09T09:28:59.087 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 55222bc9-aae0-4671-b726-8f173f4fcd7d 1 2019-12-09T09:29:00.501 INFO:teuthology.orchestra.run.smithi186.stdout:1 2019-12-09T09:29:00.523 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new ca36c64e-86ec-4661-a9ea-5a56dd1ff58c 2 2019-12-09T09:29:01.655 INFO:teuthology.orchestra.run.smithi186.stdout:2 2019-12-09T09:29:01.682 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 487c3db1-5e11-488d-af88-87464e74a014 3 2019-12-09T09:29:02.860 INFO:teuthology.orchestra.run.smithi186.stdout:3 2019-12-09T09:29:02.886 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new ef180909-e260-4dbc-b67f-e88d553d2cd7 4 2019-12-09T09:29:03.968 INFO:teuthology.orchestra.run.smithi186.stdout:4 2019-12-09T09:29:03.994 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 970602a5-f631-474b-bd44-2c13af90bbdc 5 2019-12-09T09:29:05.042 INFO:teuthology.orchestra.run.smithi186.stdout:5 2019-12-09T09:29:05.067 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 8c1a0464-37c7-43a4-8e62-afab287303fa 6 2019-12-09T09:29:06.190 INFO:teuthology.orchestra.run.smithi186.stdout:6 2019-12-09T09:29:06.217 INFO:teuthology.orchestra.run.smithi186:> sudo ceph --cluster ceph osd new 04fb56e7-9e3e-4db1-80eb-82ac594df99b 7 2019-12-09T09:29:07.314 INFO:teuthology.orchestra.run.smithi186.stdout:7 2019-12-09T09:29:07.337 DEBUG:teuthology.misc:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.338 INFO:tasks.ceph.osd.0:Restarting daemon 2019-12-09T09:29:07.338 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 0 2019-12-09T09:29:07.342 INFO:tasks.ceph.osd.0:Started 2019-12-09T09:29:07.342 DEBUG:teuthology.misc:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.342 INFO:tasks.ceph.osd.1:Restarting daemon 2019-12-09T09:29:07.343 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1 2019-12-09T09:29:07.346 INFO:tasks.ceph.osd.1:Started 2019-12-09T09:29:07.347 DEBUG:teuthology.misc:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.347 INFO:tasks.ceph.osd.2:Restarting daemon 2019-12-09T09:29:07.348 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 2 2019-12-09T09:29:07.353 INFO:tasks.ceph.osd.2:Started 2019-12-09T09:29:07.353 DEBUG:teuthology.misc:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.354 INFO:tasks.ceph.osd.3:Restarting daemon 2019-12-09T09:29:07.354 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 3 2019-12-09T09:29:07.359 INFO:tasks.ceph.osd.3:Started 2019-12-09T09:29:07.359 DEBUG:teuthology.misc:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.359 INFO:tasks.ceph.osd.4:Restarting daemon 2019-12-09T09:29:07.360 INFO:teuthology.orchestra.run.smithi186:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 4 2019-12-09T09:29:07.365 INFO:tasks.ceph.osd.4:Started 2019-12-09T09:29:07.366 DEBUG:teuthology.misc:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.366 INFO:tasks.ceph.osd.5:Restarting daemon 2019-12-09T09:29:07.366 INFO:teuthology.orchestra.run.smithi186:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 5 2019-12-09T09:29:07.369 INFO:tasks.ceph.osd.5:Started 2019-12-09T09:29:07.369 DEBUG:teuthology.misc:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.369 INFO:tasks.ceph.osd.6:Restarting daemon 2019-12-09T09:29:07.370 INFO:teuthology.orchestra.run.smithi186:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 6 2019-12-09T09:29:07.373 INFO:tasks.ceph.osd.6:Started 2019-12-09T09:29:07.373 DEBUG:teuthology.misc:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T09:29:07.373 INFO:tasks.ceph.osd.7:Restarting daemon 2019-12-09T09:29:07.373 INFO:teuthology.orchestra.run.smithi186:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 7 2019-12-09T09:29:07.376 INFO:tasks.ceph.osd.7:Started 2019-12-09T09:29:07.377 INFO:tasks.ceph:Waiting for OSDs to come up 2019-12-09T09:29:08.190 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:29:08.846 INFO:teuthology.misc.health.smithi033.stdout: 2019-12-09T09:29:08.846 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"a3644d3a-627a-4518-94ad-53a356f98d9f","created":"2019-12-09T09:28:52.478057+0000","modified":"2019-12-09T09:29:07.123790+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"6f8f10c0-106f-478a-a436-88ac562a6b42","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"55222bc9-aae0-4671-b726-8f173f4fcd7d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ca36c64e-86ec-4661-a9ea-5a56dd1ff58c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"487c3db1-5e11-488d-af88-87464e74a014","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ef180909-e260-4dbc-b67f-e88d553d2cd7","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"970602a5-f631-474b-bd44-2c13af90bbdc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8c1a0464-37c7-43a4-8e62-afab287303fa","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"04fb56e7-9e3e-4db1-80eb-82ac594df99b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:29:08.857 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-09T09:29:11.119 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:03.603 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.170 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:03.661 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.214 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:03.701 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.223 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:03.707 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.265 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:03.796 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.276 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:03.759 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.306 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:03.790 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.319 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:03.853 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.329 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:03.813 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.345 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:03.867 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.384 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:03.868 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.392 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:03.909 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.402 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:03.924 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.451 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:03.968 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.455 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:03.988 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.512 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:04.046 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:11.817 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T09:29:11.793+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:11.901 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T09:29:11.887+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:11.904 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T09:29:11.891+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:11.934 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T09:29:11.913+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:11.990 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T09:29:11.976+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:12.049 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T09:29:12.025+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:12.057 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:04.548 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.066 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T09:29:12.047+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:12.103 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:04.591 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.115 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:04.599 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.123 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T09:29:12.106+0000 9b13800 -1 Falling back to public interface 2019-12-09T09:29:12.175 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:04.709 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.199 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:04.684 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.290 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:04.807 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.294 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:04.816 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.359 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:04.893 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.463 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:04.954 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.476 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:04.967 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.505 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:04.992 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.515 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:04.999 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.518 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:05.006 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.527 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:05.011 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.593 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:05.127 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.601 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:05.085 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.606 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:05.141 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.613 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:05.097 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.683 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:05.200 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.695 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:05.212 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.696 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:05.218 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.709 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:05.231 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.762 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:05.297 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:12.774 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:05.309 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.163 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:07.654 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.173 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:07.664 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.439 INFO:tasks.ceph.osd.5.smithi186.stderr:==00:00:00:07.930 10129== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.540 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:08.023 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.550 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:08.033 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.553 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:08.040 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.564 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:08.051 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.604 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:08.120 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.614 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:08.131 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.661 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:29:15.729 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:08.264 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.734 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:08.269 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.800 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:08.322 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.805 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:08.327 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.822 INFO:tasks.ceph.osd.6.smithi186.stderr:==00:00:00:08.306 10132== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.827 INFO:tasks.ceph.osd.4.smithi186.stderr:==00:00:00:08.315 10130== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:15.876 INFO:tasks.ceph.osd.2.smithi033.stderr:==00:00:00:08.393 10163== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.004 INFO:tasks.ceph.osd.1.smithi033.stderr:==00:00:00:08.539 10160== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.060 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:08.544 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.064 INFO:tasks.ceph.osd.0.smithi033.stderr:==00:00:00:08.587 10162== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.076 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:08.560 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.202 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:08.737 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.210 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:08.745 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.344 INFO:tasks.ceph.osd.7.smithi186.stderr:==00:00:00:08.828 10131== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:16.421 INFO:teuthology.misc.health.smithi033.stdout: 2019-12-09T09:29:16.421 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":10,"fsid":"a3644d3a-627a-4518-94ad-53a356f98d9f","created":"2019-12-09T09:28:52.478057+0000","modified":"2019-12-09T09:29:07.123790+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"6f8f10c0-106f-478a-a436-88ac562a6b42","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"55222bc9-aae0-4671-b726-8f173f4fcd7d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ca36c64e-86ec-4661-a9ea-5a56dd1ff58c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"487c3db1-5e11-488d-af88-87464e74a014","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ef180909-e260-4dbc-b67f-e88d553d2cd7","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"970602a5-f631-474b-bd44-2c13af90bbdc","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"8c1a0464-37c7-43a4-8e62-afab287303fa","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"04fb56e7-9e3e-4db1-80eb-82ac594df99b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:29:16.436 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-09T09:29:16.500 INFO:tasks.ceph.osd.3.smithi033.stderr:==00:00:00:09.033 10161== Warning: unimplemented fcntl command: 1038 2019-12-09T09:29:17.035 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T09:29:17.032+0000 9b13800 -1 osd.5 0 log_to_monitors {default=true} 2019-12-09T09:29:17.203 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T09:29:17.201+0000 9b13800 -1 osd.6 0 log_to_monitors {default=true} 2019-12-09T09:29:17.244 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T09:29:17.244+0000 9b13800 -1 osd.4 0 log_to_monitors {default=true} 2019-12-09T09:29:17.280 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T09:29:17.277+0000 9b13800 -1 osd.2 0 log_to_monitors {default=true} 2019-12-09T09:29:17.493 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T09:29:17.485+0000 9b13800 -1 osd.1 0 log_to_monitors {default=true} 2019-12-09T09:29:17.618 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T09:29:17.616+0000 9b13800 -1 osd.0 0 log_to_monitors {default=true} 2019-12-09T09:29:17.759 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T09:29:17.757+0000 9b13800 -1 osd.7 0 log_to_monitors {default=true} 2019-12-09T09:29:17.917 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T09:29:17.916+0000 9b13800 -1 osd.3 0 log_to_monitors {default=true} 2019-12-09T09:29:19.152 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T09:29:19.134+0000 27772700 -1 osd.2 0 waiting for initial osdmap 2019-12-09T09:29:19.208 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T09:29:19.207+0000 2476e700 -1 osd.4 0 waiting for initial osdmap 2019-12-09T09:29:19.208 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T09:29:19.177+0000 2796d700 -1 osd.6 0 waiting for initial osdmap 2019-12-09T09:29:19.265 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T09:29:19.238+0000 2476e700 -1 osd.5 0 waiting for initial osdmap 2019-12-09T09:29:20.565 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T09:29:20.559+0000 2476e700 -1 osd.0 0 waiting for initial osdmap 2019-12-09T09:29:20.597 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T09:29:20.594+0000 2476e700 -1 osd.3 0 waiting for initial osdmap 2019-12-09T09:29:20.610 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T09:29:20.603+0000 27772700 -1 osd.1 0 waiting for initial osdmap 2019-12-09T09:29:20.613 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T09:29:20.609+0000 27772700 -1 osd.7 0 waiting for initial osdmap 2019-12-09T09:29:23.247 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T09:29:23.958 INFO:teuthology.misc.health.smithi033.stdout: 2019-12-09T09:29:23.958 INFO:teuthology.misc.health.smithi033.stdout:{"epoch":15,"fsid":"a3644d3a-627a-4518-94ad-53a356f98d9f","created":"2019-12-09T09:28:52.478057+0000","modified":"2019-12-09T09:29:22.387112+0000","last_up_change":"2019-12-09T09:29:22.387112+0000","last_in_change":"2019-12-09T09:29:22.387112+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"6f8f10c0-106f-478a-a436-88ac562a6b42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":10162},{"type":"v1","addr":"172.21.15.33:6811","nonce":10162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":10162},{"type":"v1","addr":"172.21.15.33:6815","nonce":10162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":10162},{"type":"v1","addr":"172.21.15.33:6822","nonce":10162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":10162},{"type":"v1","addr":"172.21.15.33:6818","nonce":10162}]},"public_addr":"172.21.15.33:6811/10162","cluster_addr":"172.21.15.33:6815/10162","heartbeat_back_addr":"172.21.15.33:6822/10162","heartbeat_front_addr":"172.21.15.33:6818/10162","state":["exists","up"]},{"osd":1,"uuid":"55222bc9-aae0-4671-b726-8f173f4fcd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":10160},{"type":"v1","addr":"172.21.15.33:6827","nonce":10160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":10160},{"type":"v1","addr":"172.21.15.33:6829","nonce":10160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":10160},{"type":"v1","addr":"172.21.15.33:6833","nonce":10160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":10160},{"type":"v1","addr":"172.21.15.33:6831","nonce":10160}]},"public_addr":"172.21.15.33:6827/10160","cluster_addr":"172.21.15.33:6829/10160","heartbeat_back_addr":"172.21.15.33:6833/10160","heartbeat_front_addr":"172.21.15.33:6831/10160","state":["exists","up"]},{"osd":2,"uuid":"ca36c64e-86ec-4661-a9ea-5a56dd1ff58c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":10163},{"type":"v1","addr":"172.21.15.33:6814","nonce":10163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":10163},{"type":"v1","addr":"172.21.15.33:6819","nonce":10163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":10163},{"type":"v1","addr":"172.21.15.33:6825","nonce":10163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":10163},{"type":"v1","addr":"172.21.15.33:6823","nonce":10163}]},"public_addr":"172.21.15.33:6814/10163","cluster_addr":"172.21.15.33:6819/10163","heartbeat_back_addr":"172.21.15.33:6825/10163","heartbeat_front_addr":"172.21.15.33:6823/10163","state":["exists","up"]},{"osd":3,"uuid":"487c3db1-5e11-488d-af88-87464e74a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":10161},{"type":"v1","addr":"172.21.15.33:6803","nonce":10161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":10161},{"type":"v1","addr":"172.21.15.33:6805","nonce":10161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":10161},{"type":"v1","addr":"172.21.15.33:6809","nonce":10161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":10161},{"type":"v1","addr":"172.21.15.33:6807","nonce":10161}]},"public_addr":"172.21.15.33:6803/10161","cluster_addr":"172.21.15.33:6805/10161","heartbeat_back_addr":"172.21.15.33:6809/10161","heartbeat_front_addr":"172.21.15.33:6807/10161","state":["exists","up"]},{"osd":4,"uuid":"ef180909-e260-4dbc-b67f-e88d553d2cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":10130},{"type":"v1","addr":"172.21.15.186:6809","nonce":10130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":10130},{"type":"v1","addr":"172.21.15.186:6813","nonce":10130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":10130},{"type":"v1","addr":"172.21.15.186:6819","nonce":10130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":10130},{"type":"v1","addr":"172.21.15.186:6816","nonce":10130}]},"public_addr":"172.21.15.186:6809/10130","cluster_addr":"172.21.15.186:6813/10130","heartbeat_back_addr":"172.21.15.186:6819/10130","heartbeat_front_addr":"172.21.15.186:6816/10130","state":["exists","up"]},{"osd":5,"uuid":"970602a5-f631-474b-bd44-2c13af90bbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":10129},{"type":"v1","addr":"172.21.15.186:6801","nonce":10129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":10129},{"type":"v1","addr":"172.21.15.186:6803","nonce":10129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":10129},{"type":"v1","addr":"172.21.15.186:6807","nonce":10129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":10129},{"type":"v1","addr":"172.21.15.186:6805","nonce":10129}]},"public_addr":"172.21.15.186:6801/10129","cluster_addr":"172.21.15.186:6803/10129","heartbeat_back_addr":"172.21.15.186:6807/10129","heartbeat_front_addr":"172.21.15.186:6805/10129","state":["exists","up"]},{"osd":6,"uuid":"8c1a0464-37c7-43a4-8e62-afab287303fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":10132},{"type":"v1","addr":"172.21.15.186:6811","nonce":10132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6815","nonce":10132},{"type":"v1","addr":"172.21.15.186:6817","nonce":10132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":10132},{"type":"v1","addr":"172.21.15.186:6823","nonce":10132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":10132},{"type":"v1","addr":"172.21.15.186:6821","nonce":10132}]},"public_addr":"172.21.15.186:6811/10132","cluster_addr":"172.21.15.186:6817/10132","heartbeat_back_addr":"172.21.15.186:6823/10132","heartbeat_front_addr":"172.21.15.186:6821/10132","state":["exists","up"]},{"osd":7,"uuid":"04fb56e7-9e3e-4db1-80eb-82ac594df99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":10131},{"type":"v1","addr":"172.21.15.186:6825","nonce":10131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":10131},{"type":"v1","addr":"172.21.15.186:6827","nonce":10131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":10131},{"type":"v1","addr":"172.21.15.186:6831","nonce":10131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":10131},{"type":"v1","addr":"172.21.15.186:6829","nonce":10131}]},"public_addr":"172.21.15.186:6825/10131","cluster_addr":"172.21.15.186:6827/10131","heartbeat_back_addr":"172.21.15.186:6831/10131","heartbeat_front_addr":"172.21.15.186:6829/10131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.702739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.403410+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.997415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.357249+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.079046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.328793+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.909759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:29:23.976 DEBUG:teuthology.misc:8 of 8 OSDs are up 2019-12-09T09:29:23.977 INFO:tasks.ceph:Creating RBD pool 2019-12-09T09:29:23.977 INFO:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool create rbd 8 2019-12-09T09:29:25.066 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'rbd' created 2019-12-09T09:29:25.084 INFO:teuthology.orchestra.run.smithi033:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2019-12-09T09:29:26.351 INFO:teuthology.orchestra.run.smithi033.stderr:enabled application 'rbd' on pool 'rbd' 2019-12-09T09:29:26.365 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2019-12-09T09:29:26.366 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:29:26.366 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2019-12-09T09:29:27.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:29:27.086 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"fsid":"a3644d3a-627a-4518-94ad-53a356f98d9f","created":"2019-12-09T09:28:52.478057+0000","modified":"2019-12-09T09:29:26.226150+0000","last_up_change":"2019-12-09T09:29:22.387112+0000","last_in_change":"2019-12-09T09:29:22.387112+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:29:24.726523+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6f8f10c0-106f-478a-a436-88ac562a6b42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":10162},{"type":"v1","addr":"172.21.15.33:6811","nonce":10162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":10162},{"type":"v1","addr":"172.21.15.33:6815","nonce":10162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":10162},{"type":"v1","addr":"172.21.15.33:6822","nonce":10162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":10162},{"type":"v1","addr":"172.21.15.33:6818","nonce":10162}]},"public_addr":"172.21.15.33:6811/10162","cluster_addr":"172.21.15.33:6815/10162","heartbeat_back_addr":"172.21.15.33:6822/10162","heartbeat_front_addr":"172.21.15.33:6818/10162","state":["exists","up"]},{"osd":1,"uuid":"55222bc9-aae0-4671-b726-8f173f4fcd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":10160},{"type":"v1","addr":"172.21.15.33:6827","nonce":10160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":10160},{"type":"v1","addr":"172.21.15.33:6829","nonce":10160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":10160},{"type":"v1","addr":"172.21.15.33:6833","nonce":10160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":10160},{"type":"v1","addr":"172.21.15.33:6831","nonce":10160}]},"public_addr":"172.21.15.33:6827/10160","cluster_addr":"172.21.15.33:6829/10160","heartbeat_back_addr":"172.21.15.33:6833/10160","heartbeat_front_addr":"172.21.15.33:6831/10160","state":["exists","up"]},{"osd":2,"uuid":"ca36c64e-86ec-4661-a9ea-5a56dd1ff58c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":10163},{"type":"v1","addr":"172.21.15.33:6814","nonce":10163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":10163},{"type":"v1","addr":"172.21.15.33:6819","nonce":10163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":10163},{"type":"v1","addr":"172.21.15.33:6825","nonce":10163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":10163},{"type":"v1","addr":"172.21.15.33:6823","nonce":10163}]},"public_addr":"172.21.15.33:6814/10163","cluster_addr":"172.21.15.33:6819/10163","heartbeat_back_addr":"172.21.15.33:6825/10163","heartbeat_front_addr":"172.21.15.33:6823/10163","state":["exists","up"]},{"osd":3,"uuid":"487c3db1-5e11-488d-af88-87464e74a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":10161},{"type":"v1","addr":"172.21.15.33:6803","nonce":10161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":10161},{"type":"v1","addr":"172.21.15.33:6805","nonce":10161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":10161},{"type":"v1","addr":"172.21.15.33:6809","nonce":10161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":10161},{"type":"v1","addr":"172.21.15.33:6807","nonce":10161}]},"public_addr":"172.21.15.33:6803/10161","cluster_addr":"172.21.15.33:6805/10161","heartbeat_back_addr":"172.21.15.33:6809/10161","heartbeat_front_addr":"172.21.15.33:6807/10161","state":["exists","up"]},{"osd":4,"uuid":"ef180909-e260-4dbc-b67f-e88d553d2cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":10130},{"type":"v1","addr":"172.21.15.186:6809","nonce":10130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":10130},{"type":"v1","addr":"172.21.15.186:6813","nonce":10130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":10130},{"type":"v1","addr":"172.21.15.186:6819","nonce":10130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":10130},{"type":"v1","addr":"172.21.15.186:6816","nonce":10130}]},"public_addr":"172.21.15.186:6809/10130","cluster_addr":"172.21.15.186:6813/10130","heartbeat_back_addr":"172.21.15.186:6819/10130","heartbeat_front_addr":"172.21.15.186:6816/10130","state":["exists","up"]},{"osd":5,"uuid":"970602a5-f631-474b-bd44-2c13af90bbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":10129},{"type":"v1","addr":"172.21.15.186:6801","nonce":10129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":10129},{"type":"v1","addr":"172.21.15.186:6803","nonce":10129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":10129},{"type":"v1","addr":"172.21.15.186:6807","nonce":10129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":10129},{"type":"v1","addr":"172.21.15.186:6805","nonce":10129}]},"public_addr":"172.21.15.186:6801/10129","cluster_addr":"172.21.15.186:6803/10129","heartbeat_back_addr":"172.21.15.186:6807/10129","heartbeat_front_addr":"172.21.15.186:6805/10129","state":["exists","up"]},{"osd":6,"uuid":"8c1a0464-37c7-43a4-8e62-afab287303fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":10132},{"type":"v1","addr":"172.21.15.186:6811","nonce":10132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6815","nonce":10132},{"type":"v1","addr":"172.21.15.186:6817","nonce":10132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":10132},{"type":"v1","addr":"172.21.15.186:6823","nonce":10132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":10132},{"type":"v1","addr":"172.21.15.186:6821","nonce":10132}]},"public_addr":"172.21.15.186:6811/10132","cluster_addr":"172.21.15.186:6817/10132","heartbeat_back_addr":"172.21.15.186:6823/10132","heartbeat_front_addr":"172.21.15.186:6821/10132","state":["exists","up"]},{"osd":7,"uuid":"04fb56e7-9e3e-4db1-80eb-82ac594df99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":10131},{"type":"v1","addr":"172.21.15.186:6825","nonce":10131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":10131},{"type":"v1","addr":"172.21.15.186:6827","nonce":10131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":10131},{"type":"v1","addr":"172.21.15.186:6831","nonce":10131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":10131},{"type":"v1","addr":"172.21.15.186:6829","nonce":10131}]},"public_addr":"172.21.15.186:6825/10131","cluster_addr":"172.21.15.186:6827/10131","heartbeat_back_addr":"172.21.15.186:6831/10131","heartbeat_front_addr":"172.21.15.186:6829/10131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.702739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.567636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.403410+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.997415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.357249+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.079046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.328793+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.909759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:29:27.098 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2019-12-09T09:29:24.726523+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'17', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2019-12-09T09:29:27.099 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2019-12-09T09:29:27.834 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 8 2019-12-09T09:29:27.848 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-12-09T09:29:27.848 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2019-12-09T09:29:27.848 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2019-12-09T09:29:28.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:29:28.712 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"active_gid":4105,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":9135},{"type":"v1","addr":"172.21.15.33:6801","nonce":9135}]},"active_addr":"172.21.15.33:6801/9135","active_change":"2019-12-09T09:28:56.348566+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]}} 2019-12-09T09:29:28.723 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2019-12-09T09:29:28.723 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2019-12-09T09:29:28.724 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:29:29.540 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:29:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"fsid":"a3644d3a-627a-4518-94ad-53a356f98d9f","created":"2019-12-09T09:28:52.478057+0000","modified":"2019-12-09T09:29:28.624244+0000","last_up_change":"2019-12-09T09:29:22.387112+0000","last_in_change":"2019-12-09T09:29:22.387112+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:29:24.726523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6f8f10c0-106f-478a-a436-88ac562a6b42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":10162},{"type":"v1","addr":"172.21.15.33:6811","nonce":10162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":10162},{"type":"v1","addr":"172.21.15.33:6815","nonce":10162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":10162},{"type":"v1","addr":"172.21.15.33:6822","nonce":10162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":10162},{"type":"v1","addr":"172.21.15.33:6818","nonce":10162}]},"public_addr":"172.21.15.33:6811/10162","cluster_addr":"172.21.15.33:6815/10162","heartbeat_back_addr":"172.21.15.33:6822/10162","heartbeat_front_addr":"172.21.15.33:6818/10162","state":["exists","up"]},{"osd":1,"uuid":"55222bc9-aae0-4671-b726-8f173f4fcd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":10160},{"type":"v1","addr":"172.21.15.33:6827","nonce":10160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":10160},{"type":"v1","addr":"172.21.15.33:6829","nonce":10160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":10160},{"type":"v1","addr":"172.21.15.33:6833","nonce":10160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":10160},{"type":"v1","addr":"172.21.15.33:6831","nonce":10160}]},"public_addr":"172.21.15.33:6827/10160","cluster_addr":"172.21.15.33:6829/10160","heartbeat_back_addr":"172.21.15.33:6833/10160","heartbeat_front_addr":"172.21.15.33:6831/10160","state":["exists","up"]},{"osd":2,"uuid":"ca36c64e-86ec-4661-a9ea-5a56dd1ff58c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":10163},{"type":"v1","addr":"172.21.15.33:6814","nonce":10163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":10163},{"type":"v1","addr":"172.21.15.33:6819","nonce":10163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":10163},{"type":"v1","addr":"172.21.15.33:6825","nonce":10163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":10163},{"type":"v1","addr":"172.21.15.33:6823","nonce":10163}]},"public_addr":"172.21.15.33:6814/10163","cluster_addr":"172.21.15.33:6819/10163","heartbeat_back_addr":"172.21.15.33:6825/10163","heartbeat_front_addr":"172.21.15.33:6823/10163","state":["exists","up"]},{"osd":3,"uuid":"487c3db1-5e11-488d-af88-87464e74a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":10161},{"type":"v1","addr":"172.21.15.33:6803","nonce":10161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":10161},{"type":"v1","addr":"172.21.15.33:6805","nonce":10161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":10161},{"type":"v1","addr":"172.21.15.33:6809","nonce":10161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":10161},{"type":"v1","addr":"172.21.15.33:6807","nonce":10161}]},"public_addr":"172.21.15.33:6803/10161","cluster_addr":"172.21.15.33:6805/10161","heartbeat_back_addr":"172.21.15.33:6809/10161","heartbeat_front_addr":"172.21.15.33:6807/10161","state":["exists","up"]},{"osd":4,"uuid":"ef180909-e260-4dbc-b67f-e88d553d2cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":10130},{"type":"v1","addr":"172.21.15.186:6809","nonce":10130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":10130},{"type":"v1","addr":"172.21.15.186:6813","nonce":10130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":10130},{"type":"v1","addr":"172.21.15.186:6819","nonce":10130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":10130},{"type":"v1","addr":"172.21.15.186:6816","nonce":10130}]},"public_addr":"172.21.15.186:6809/10130","cluster_addr":"172.21.15.186:6813/10130","heartbeat_back_addr":"172.21.15.186:6819/10130","heartbeat_front_addr":"172.21.15.186:6816/10130","state":["exists","up"]},{"osd":5,"uuid":"970602a5-f631-474b-bd44-2c13af90bbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":10129},{"type":"v1","addr":"172.21.15.186:6801","nonce":10129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":10129},{"type":"v1","addr":"172.21.15.186:6803","nonce":10129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":10129},{"type":"v1","addr":"172.21.15.186:6807","nonce":10129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":10129},{"type":"v1","addr":"172.21.15.186:6805","nonce":10129}]},"public_addr":"172.21.15.186:6801/10129","cluster_addr":"172.21.15.186:6803/10129","heartbeat_back_addr":"172.21.15.186:6807/10129","heartbeat_front_addr":"172.21.15.186:6805/10129","state":["exists","up"]},{"osd":6,"uuid":"8c1a0464-37c7-43a4-8e62-afab287303fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":10132},{"type":"v1","addr":"172.21.15.186:6811","nonce":10132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6815","nonce":10132},{"type":"v1","addr":"172.21.15.186:6817","nonce":10132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":10132},{"type":"v1","addr":"172.21.15.186:6823","nonce":10132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":10132},{"type":"v1","addr":"172.21.15.186:6821","nonce":10132}]},"public_addr":"172.21.15.186:6811/10132","cluster_addr":"172.21.15.186:6817/10132","heartbeat_back_addr":"172.21.15.186:6823/10132","heartbeat_front_addr":"172.21.15.186:6821/10132","state":["exists","up"]},{"osd":7,"uuid":"04fb56e7-9e3e-4db1-80eb-82ac594df99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":10131},{"type":"v1","addr":"172.21.15.186:6825","nonce":10131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":10131},{"type":"v1","addr":"172.21.15.186:6827","nonce":10131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":10131},{"type":"v1","addr":"172.21.15.186:6831","nonce":10131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":10131},{"type":"v1","addr":"172.21.15.186:6829","nonce":10131}]},"public_addr":"172.21.15.186:6825/10131","cluster_addr":"172.21.15.186:6827/10131","heartbeat_back_addr":"172.21.15.186:6831/10131","heartbeat_front_addr":"172.21.15.186:6829/10131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.702739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.567636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.403410+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.997415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.357249+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.079046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.328793+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.909759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:29:29.563 INFO:tasks.ceph.ceph_manager.ceph:all up! 2019-12-09T09:29:29.564 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T09:29:30.494 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:29:30.495 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"fsid":"a3644d3a-627a-4518-94ad-53a356f98d9f","created":"2019-12-09T09:28:52.478057+0000","modified":"2019-12-09T09:29:28.624244+0000","last_up_change":"2019-12-09T09:29:22.387112+0000","last_in_change":"2019-12-09T09:29:22.387112+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T09:29:24.726523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6f8f10c0-106f-478a-a436-88ac562a6b42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":10162},{"type":"v1","addr":"172.21.15.33:6811","nonce":10162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6813","nonce":10162},{"type":"v1","addr":"172.21.15.33:6815","nonce":10162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":10162},{"type":"v1","addr":"172.21.15.33:6822","nonce":10162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":10162},{"type":"v1","addr":"172.21.15.33:6818","nonce":10162}]},"public_addr":"172.21.15.33:6811/10162","cluster_addr":"172.21.15.33:6815/10162","heartbeat_back_addr":"172.21.15.33:6822/10162","heartbeat_front_addr":"172.21.15.33:6818/10162","state":["exists","up"]},{"osd":1,"uuid":"55222bc9-aae0-4671-b726-8f173f4fcd7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":10160},{"type":"v1","addr":"172.21.15.33:6827","nonce":10160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":10160},{"type":"v1","addr":"172.21.15.33:6829","nonce":10160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":10160},{"type":"v1","addr":"172.21.15.33:6833","nonce":10160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":10160},{"type":"v1","addr":"172.21.15.33:6831","nonce":10160}]},"public_addr":"172.21.15.33:6827/10160","cluster_addr":"172.21.15.33:6829/10160","heartbeat_back_addr":"172.21.15.33:6833/10160","heartbeat_front_addr":"172.21.15.33:6831/10160","state":["exists","up"]},{"osd":2,"uuid":"ca36c64e-86ec-4661-a9ea-5a56dd1ff58c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":10163},{"type":"v1","addr":"172.21.15.33:6814","nonce":10163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6817","nonce":10163},{"type":"v1","addr":"172.21.15.33:6819","nonce":10163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":10163},{"type":"v1","addr":"172.21.15.33:6825","nonce":10163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6821","nonce":10163},{"type":"v1","addr":"172.21.15.33:6823","nonce":10163}]},"public_addr":"172.21.15.33:6814/10163","cluster_addr":"172.21.15.33:6819/10163","heartbeat_back_addr":"172.21.15.33:6825/10163","heartbeat_front_addr":"172.21.15.33:6823/10163","state":["exists","up"]},{"osd":3,"uuid":"487c3db1-5e11-488d-af88-87464e74a014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":10161},{"type":"v1","addr":"172.21.15.33:6803","nonce":10161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":10161},{"type":"v1","addr":"172.21.15.33:6805","nonce":10161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":10161},{"type":"v1","addr":"172.21.15.33:6809","nonce":10161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":10161},{"type":"v1","addr":"172.21.15.33:6807","nonce":10161}]},"public_addr":"172.21.15.33:6803/10161","cluster_addr":"172.21.15.33:6805/10161","heartbeat_back_addr":"172.21.15.33:6809/10161","heartbeat_front_addr":"172.21.15.33:6807/10161","state":["exists","up"]},{"osd":4,"uuid":"ef180909-e260-4dbc-b67f-e88d553d2cd7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6808","nonce":10130},{"type":"v1","addr":"172.21.15.186:6809","nonce":10130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6812","nonce":10130},{"type":"v1","addr":"172.21.15.186:6813","nonce":10130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6818","nonce":10130},{"type":"v1","addr":"172.21.15.186:6819","nonce":10130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6814","nonce":10130},{"type":"v1","addr":"172.21.15.186:6816","nonce":10130}]},"public_addr":"172.21.15.186:6809/10130","cluster_addr":"172.21.15.186:6813/10130","heartbeat_back_addr":"172.21.15.186:6819/10130","heartbeat_front_addr":"172.21.15.186:6816/10130","state":["exists","up"]},{"osd":5,"uuid":"970602a5-f631-474b-bd44-2c13af90bbdc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6800","nonce":10129},{"type":"v1","addr":"172.21.15.186:6801","nonce":10129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6802","nonce":10129},{"type":"v1","addr":"172.21.15.186:6803","nonce":10129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6806","nonce":10129},{"type":"v1","addr":"172.21.15.186:6807","nonce":10129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6804","nonce":10129},{"type":"v1","addr":"172.21.15.186:6805","nonce":10129}]},"public_addr":"172.21.15.186:6801/10129","cluster_addr":"172.21.15.186:6803/10129","heartbeat_back_addr":"172.21.15.186:6807/10129","heartbeat_front_addr":"172.21.15.186:6805/10129","state":["exists","up"]},{"osd":6,"uuid":"8c1a0464-37c7-43a4-8e62-afab287303fa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6810","nonce":10132},{"type":"v1","addr":"172.21.15.186:6811","nonce":10132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6815","nonce":10132},{"type":"v1","addr":"172.21.15.186:6817","nonce":10132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6822","nonce":10132},{"type":"v1","addr":"172.21.15.186:6823","nonce":10132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6820","nonce":10132},{"type":"v1","addr":"172.21.15.186:6821","nonce":10132}]},"public_addr":"172.21.15.186:6811/10132","cluster_addr":"172.21.15.186:6817/10132","heartbeat_back_addr":"172.21.15.186:6823/10132","heartbeat_front_addr":"172.21.15.186:6821/10132","state":["exists","up"]},{"osd":7,"uuid":"04fb56e7-9e3e-4db1-80eb-82ac594df99b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6824","nonce":10131},{"type":"v1","addr":"172.21.15.186:6825","nonce":10131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6826","nonce":10131},{"type":"v1","addr":"172.21.15.186:6827","nonce":10131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6830","nonce":10131},{"type":"v1","addr":"172.21.15.186:6831","nonce":10131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.186:6828","nonce":10131},{"type":"v1","addr":"172.21.15.186:6829","nonce":10131}]},"public_addr":"172.21.15.186:6825/10131","cluster_addr":"172.21.15.186:6827/10131","heartbeat_back_addr":"172.21.15.186:6831/10131","heartbeat_front_addr":"172.21.15.186:6829/10131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.702739+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.567636+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.403410+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.997415+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.357249+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.079046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.328793+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T09:29:18.909759+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T09:29:30.518 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2019-12-09T09:29:30.987 INFO:teuthology.orchestra.run.smithi033.stdout:60129542148 2019-12-09T09:29:30.987 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2019-12-09T09:29:31.514 INFO:teuthology.orchestra.run.smithi033.stdout:64424509446 2019-12-09T09:29:31.514 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2019-12-09T09:29:31.977 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2019-12-09T09:29:31.978 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2019-12-09T09:29:32.527 INFO:teuthology.orchestra.run.smithi033.stdout:60129542149 2019-12-09T09:29:32.527 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2019-12-09T09:29:33.072 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2019-12-09T09:29:33.072 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2019-12-09T09:29:33.775 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2019-12-09T09:29:33.775 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2019-12-09T09:29:34.448 INFO:teuthology.orchestra.run.smithi033.stdout:55834574854 2019-12-09T09:29:34.449 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2019-12-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi033.stdout:60129542148 2019-12-09T09:29:35.045 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2019-12-09T09:29:35.954 INFO:teuthology.orchestra.run.smithi033.stdout:60129542149 2019-12-09T09:29:35.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.0 2019-12-09T09:29:35.968 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2019-12-09T09:29:36.799 INFO:teuthology.orchestra.run.smithi033.stdout:64424509447 2019-12-09T09:29:36.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.1 2019-12-09T09:29:36.813 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2019-12-09T09:29:37.569 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2019-12-09T09:29:37.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.2 2019-12-09T09:29:37.579 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2019-12-09T09:29:38.320 INFO:teuthology.orchestra.run.smithi033.stdout:60129542149 2019-12-09T09:29:38.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2019-12-09T09:29:38.334 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2019-12-09T09:29:39.071 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2019-12-09T09:29:39.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.4 2019-12-09T09:29:39.084 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2019-12-09T09:29:39.915 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2019-12-09T09:29:39.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2019-12-09T09:29:39.930 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2019-12-09T09:29:40.797 INFO:teuthology.orchestra.run.smithi033.stdout:55834574855 2019-12-09T09:29:40.810 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.6 2019-12-09T09:29:40.811 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2019-12-09T09:29:41.637 INFO:teuthology.orchestra.run.smithi033.stdout:60129542149 2019-12-09T09:29:41.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.7 2019-12-09T09:29:41.653 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2019-12-09T09:29:41.653 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T09:29:42.369 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2019-12-09T09:29:42.370 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:29:42.392 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2019-12-09T09:29:40.509269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390144,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584576,"statfs":{"total":773094113280,"available":764502605824,"internally_reserved":8589934592,"allocated":1507328,"data_stored":476344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.369985+0000","last_change":"2019-12-09T09:29:28.348293+0000","last_active":"2019-12-09T09:29:28.369985+0000","last_peered":"2019-12-09T09:29:28.369985+0000","last_clean":"2019-12-09T09:29:28.369985+0000","last_became_active":"2019-12-09T09:29:28.331346+0000","last_became_peered":"2019-12-09T09:29:28.331346+0000","last_unstale":"2019-12-09T09:29:28.369985+0000","last_undegraded":"2019-12-09T09:29:28.369985+0000","last_fullsized":"2019-12-09T09:29:28.369985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.233189+0000","last_change":"2019-12-09T09:29:27.039107+0000","last_active":"2019-12-09T09:29:28.233189+0000","last_peered":"2019-12-09T09:29:28.233189+0000","last_clean":"2019-12-09T09:29:28.233189+0000","last_became_active":"2019-12-09T09:29:26.999145+0000","last_became_peered":"2019-12-09T09:29:26.999145+0000","last_unstale":"2019-12-09T09:29:28.233189+0000","last_undegraded":"2019-12-09T09:29:28.233189+0000","last_fullsized":"2019-12-09T09:29:28.233189+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:27.729185+0000","last_change":"2019-12-09T09:29:26.993131+0000","last_active":"2019-12-09T09:29:27.729185+0000","last_peered":"2019-12-09T09:29:27.729185+0000","last_clean":"2019-12-09T09:29:27.729185+0000","last_became_active":"2019-12-09T09:29:26.956584+0000","last_became_peered":"2019-12-09T09:29:26.956584+0000","last_unstale":"2019-12-09T09:29:27.729185+0000","last_undegraded":"2019-12-09T09:29:27.729185+0000","last_fullsized":"2019-12-09T09:29:27.729185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.028150+0000","last_change":"2019-12-09T09:29:28.000852+0000","last_active":"2019-12-09T09:29:28.028150+0000","last_peered":"2019-12-09T09:29:28.028150+0000","last_clean":"2019-12-09T09:29:28.028150+0000","last_became_active":"2019-12-09T09:29:27.960997+0000","last_became_peered":"2019-12-09T09:29:27.960997+0000","last_unstale":"2019-12-09T09:29:28.028150+0000","last_undegraded":"2019-12-09T09:29:28.028150+0000","last_fullsized":"2019-12-09T09:29:28.028150+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.366887+0000","last_change":"2019-12-09T09:29:28.301651+0000","last_active":"2019-12-09T09:29:28.366887+0000","last_peered":"2019-12-09T09:29:28.366887+0000","last_clean":"2019-12-09T09:29:28.366887+0000","last_became_active":"2019-12-09T09:29:28.256859+0000","last_became_peered":"2019-12-09T09:29:28.256859+0000","last_unstale":"2019-12-09T09:29:28.366887+0000","last_undegraded":"2019-12-09T09:29:28.366887+0000","last_fullsized":"2019-12-09T09:29:28.366887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.238822+0000","last_change":"2019-12-09T09:29:27.005160+0000","last_active":"2019-12-09T09:29:28.238822+0000","last_peered":"2019-12-09T09:29:28.238822+0000","last_clean":"2019-12-09T09:29:28.238822+0000","last_became_active":"2019-12-09T09:29:26.936257+0000","last_became_peered":"2019-12-09T09:29:26.936257+0000","last_unstale":"2019-12-09T09:29:28.238822+0000","last_undegraded":"2019-12-09T09:29:28.238822+0000","last_fullsized":"2019-12-09T09:29:28.238822+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.040739+0000","last_change":"2019-12-09T09:29:28.012281+0000","last_active":"2019-12-09T09:29:28.040739+0000","last_peered":"2019-12-09T09:29:28.040739+0000","last_clean":"2019-12-09T09:29:28.040739+0000","last_became_active":"2019-12-09T09:29:27.958467+0000","last_became_peered":"2019-12-09T09:29:27.958467+0000","last_unstale":"2019-12-09T09:29:28.040739+0000","last_undegraded":"2019-12-09T09:29:28.040739+0000","last_fullsized":"2019-12-09T09:29:28.040739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.054542+0000","last_change":"2019-12-09T09:29:28.020498+0000","last_active":"2019-12-09T09:29:28.054542+0000","last_peered":"2019-12-09T09:29:28.054542+0000","last_clean":"2019-12-09T09:29:28.054542+0000","last_became_active":"2019-12-09T09:29:27.975780+0000","last_became_peered":"2019-12-09T09:29:27.975780+0000","last_unstale":"2019-12-09T09:29:28.054542+0000","last_undegraded":"2019-12-09T09:29:28.054542+0000","last_fullsized":"2019-12-09T09:29:28.054542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.400}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.578}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.840}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.554}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.364}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.864}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.847}]}]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.591}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":112.687}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.410}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.104}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.801}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.822}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.176}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.893}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.845}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.129}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.186}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.745}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.723}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.990}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.773}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.765}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.257}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.277}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.403}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.500}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.818}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.132}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.476}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.182}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":39.509}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.865}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.870}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.640}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.844}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.451}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.822}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.220}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.636}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.972}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.552}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.047}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.046}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.838}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.611}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.154}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.225}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.362}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.186}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.304}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.465}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.903}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.482}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":113.660}]}]}]}} 2019-12-09T09:29:42.396 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T09:29:43.214 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2019-12-09T09:29:43.214 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:29:43.227 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2019-12-09T09:29:42.509629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390144,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584576,"statfs":{"total":773094113280,"available":764502605824,"internally_reserved":8589934592,"allocated":1507328,"data_stored":476344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.369985+0000","last_change":"2019-12-09T09:29:28.348293+0000","last_active":"2019-12-09T09:29:28.369985+0000","last_peered":"2019-12-09T09:29:28.369985+0000","last_clean":"2019-12-09T09:29:28.369985+0000","last_became_active":"2019-12-09T09:29:28.331346+0000","last_became_peered":"2019-12-09T09:29:28.331346+0000","last_unstale":"2019-12-09T09:29:28.369985+0000","last_undegraded":"2019-12-09T09:29:28.369985+0000","last_fullsized":"2019-12-09T09:29:28.369985+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.233189+0000","last_change":"2019-12-09T09:29:27.039107+0000","last_active":"2019-12-09T09:29:28.233189+0000","last_peered":"2019-12-09T09:29:28.233189+0000","last_clean":"2019-12-09T09:29:28.233189+0000","last_became_active":"2019-12-09T09:29:26.999145+0000","last_became_peered":"2019-12-09T09:29:26.999145+0000","last_unstale":"2019-12-09T09:29:28.233189+0000","last_undegraded":"2019-12-09T09:29:28.233189+0000","last_fullsized":"2019-12-09T09:29:28.233189+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:27.729185+0000","last_change":"2019-12-09T09:29:26.993131+0000","last_active":"2019-12-09T09:29:27.729185+0000","last_peered":"2019-12-09T09:29:27.729185+0000","last_clean":"2019-12-09T09:29:27.729185+0000","last_became_active":"2019-12-09T09:29:26.956584+0000","last_became_peered":"2019-12-09T09:29:26.956584+0000","last_unstale":"2019-12-09T09:29:27.729185+0000","last_undegraded":"2019-12-09T09:29:27.729185+0000","last_fullsized":"2019-12-09T09:29:27.729185+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.028150+0000","last_change":"2019-12-09T09:29:28.000852+0000","last_active":"2019-12-09T09:29:28.028150+0000","last_peered":"2019-12-09T09:29:28.028150+0000","last_clean":"2019-12-09T09:29:28.028150+0000","last_became_active":"2019-12-09T09:29:27.960997+0000","last_became_peered":"2019-12-09T09:29:27.960997+0000","last_unstale":"2019-12-09T09:29:28.028150+0000","last_undegraded":"2019-12-09T09:29:28.028150+0000","last_fullsized":"2019-12-09T09:29:28.028150+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.366887+0000","last_change":"2019-12-09T09:29:28.301651+0000","last_active":"2019-12-09T09:29:28.366887+0000","last_peered":"2019-12-09T09:29:28.366887+0000","last_clean":"2019-12-09T09:29:28.366887+0000","last_became_active":"2019-12-09T09:29:28.256859+0000","last_became_peered":"2019-12-09T09:29:28.256859+0000","last_unstale":"2019-12-09T09:29:28.366887+0000","last_undegraded":"2019-12-09T09:29:28.366887+0000","last_fullsized":"2019-12-09T09:29:28.366887+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.238822+0000","last_change":"2019-12-09T09:29:27.005160+0000","last_active":"2019-12-09T09:29:28.238822+0000","last_peered":"2019-12-09T09:29:28.238822+0000","last_clean":"2019-12-09T09:29:28.238822+0000","last_became_active":"2019-12-09T09:29:26.936257+0000","last_became_peered":"2019-12-09T09:29:26.936257+0000","last_unstale":"2019-12-09T09:29:28.238822+0000","last_undegraded":"2019-12-09T09:29:28.238822+0000","last_fullsized":"2019-12-09T09:29:28.238822+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.040739+0000","last_change":"2019-12-09T09:29:28.012281+0000","last_active":"2019-12-09T09:29:28.040739+0000","last_peered":"2019-12-09T09:29:28.040739+0000","last_clean":"2019-12-09T09:29:28.040739+0000","last_became_active":"2019-12-09T09:29:27.958467+0000","last_became_peered":"2019-12-09T09:29:27.958467+0000","last_unstale":"2019-12-09T09:29:28.040739+0000","last_undegraded":"2019-12-09T09:29:28.040739+0000","last_fullsized":"2019-12-09T09:29:28.040739+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"18","state":"active+clean","last_fresh":"2019-12-09T09:29:28.054542+0000","last_change":"2019-12-09T09:29:28.020498+0000","last_active":"2019-12-09T09:29:28.054542+0000","last_peered":"2019-12-09T09:29:28.054542+0000","last_clean":"2019-12-09T09:29:28.054542+0000","last_became_active":"2019-12-09T09:29:27.975780+0000","last_became_peered":"2019-12-09T09:29:27.975780+0000","last_unstale":"2019-12-09T09:29:28.054542+0000","last_undegraded":"2019-12-09T09:29:28.054542+0000","last_fullsized":"2019-12-09T09:29:28.054542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T09:29:24.736379+0000","last_clean_scrub_stamp":"2019-12-09T09:29:24.736379+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.400}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.578}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.840}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.554}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.364}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.864}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.847}]}]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.531}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.213}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.401}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.905}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.117}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.670}]}]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.580}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.213}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.991}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.094}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.399}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.416}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.099}]}]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":46.192}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.704}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":43.422}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.553}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.222}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.405}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.338}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.562}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.177}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":8.394}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.983}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.236}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.123}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.711}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.693}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.624}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.331}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.449}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.449}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.077}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.207}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.701}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.707}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.552}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.718}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.805}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.030}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048768,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323072,"statfs":{"total":96636764160,"available":95562825728,"internally_reserved":1073741824,"allocated":188416,"data_stored":59543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.328}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.596}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.476}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.917}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.527}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.373}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.454}]}]}]}} 2019-12-09T09:29:43.230 INFO:tasks.ceph.ceph_manager.ceph:clean! 2019-12-09T09:29:43.230 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-12-09T09:29:43.230 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2019-12-09T09:29:43.230 INFO:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2019-12-09T09:29:43.914 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:29:43.927 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-12-09T09:29:43.928 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2019-12-09T09:29:43.928 INFO:teuthology.run_tasks:Running task openssl_keys... 2019-12-09T09:29:43.982 INFO:teuthology.run_tasks:Running task rgw... 2019-12-09T09:29:44.015 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2019-12-09T09:29:44.015 DEBUG:tasks.rgw:client list is ['client.0'] 2019-12-09T09:29:44.016 INFO:tasks.rgw:Creating data pools 2019-12-09T09:29:44.016 DEBUG:tasks.rgw:Obtaining remote for client client.0 2019-12-09T09:29:44.016 INFO:teuthology.orchestra.run.smithi033:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=2 m=1 --cluster ceph 2019-12-09T09:29:45.534 INFO:teuthology.orchestra.run.smithi033:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2019-12-09T09:29:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'default.rgw.buckets.data' created 2019-12-09T09:29:48.211 INFO:teuthology.orchestra.run.smithi033:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2019-12-09T09:29:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2019-12-09T09:29:49.154 INFO:teuthology.orchestra.run.smithi033:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2019-12-09T09:29:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'default.rgw.buckets.index' created 2019-12-09T09:29:50.489 INFO:teuthology.orchestra.run.smithi033:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2019-12-09T09:29:51.562 INFO:teuthology.orchestra.run.smithi033.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2019-12-09T09:29:51.591 DEBUG:tasks.rgw:Pools created 2019-12-09T09:29:51.591 INFO:tasks.rgw:Configuring compression type = random 2019-12-09T09:29:51.591 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T09:29:51.591 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T09:29:51.592 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T09:29:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:51.791+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:54.423+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:54.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:54.423+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:54.447+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:54.446 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:54.447+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:54.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:54.456+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:54.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:54.456+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:54.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:54.586+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:55.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.965+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:55.974 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.975+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:55.974 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.975+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.986+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:55.985 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.986+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:55.994 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.995+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.995+0000 7f24fd959ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:29:55.995 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:55.995+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.005+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:56.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.005+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:56.016 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.017+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:56.016 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.017+0000 7f24fd959ec0 10 Creating default zonegroup 2019-12-09T09:29:56.016 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.017+0000 7f24fd959ec0 20 get_system_obj_state: rctx=0x7fffe90f17a0 obj=:.pools.avail state=0x559ee7bab3f0 s->prefetch_data=0 2019-12-09T09:29:56.021 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.021+0000 7f24fd959ec0 10 couldn't find old data placement pools config, setting up new ones for the zone 2019-12-09T09:29:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.224+0000 7f24fd959ec0 10 WARNING: store->list_zones() returned r=-2 2019-12-09T09:29:56.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.224+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:56.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:56.234+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:57.365 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.365+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.370+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.370+0000 7f24fd959ec0 10 could not read realm id: (2) No such file or directory 2019-12-09T09:29:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.370+0000 7f24fd959ec0 10 WARNING: failed to set zone as default, r=-22 2019-12-09T09:29:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.370+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:57.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.377+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:57.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.922+0000 7f24eab83700 0 -- 172.21.15.33:0/1123468221 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x559ee7ba4250 msgr2=0x559ee7ba66d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:29:57.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.922+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:57.998 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.997+0000 7f24fd959ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:29:57.998 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:57.998+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:58.052 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.053+0000 7f24fd959ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:29:58.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.053+0000 7f24fd959ec0 20 zonegroup default 2019-12-09T09:29:58.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.053+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:58.063 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.064+0000 7f24fd959ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:29:58.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.064+0000 7f24fd959ec0 10 Cannot find current period zone using local zone 2019-12-09T09:29:58.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.064+0000 7f24fd959ec0 10 Using default name default 2019-12-09T09:29:58.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.064+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:58.116 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.117+0000 7f24fd959ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:29:58.117 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.117+0000 7f24fd959ec0 20 rados->read ofs=0 len=0 2019-12-09T09:29:58.134 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.134+0000 7f24fd959ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.134+0000 7f24fd959ec0 20 zone default found 2019-12-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.134+0000 7f24fd959ec0 20 started sync module instance, tier type = 2019-12-09T09:29:58.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:29:58.134+0000 7f24fd959ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:30:01.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:01.582+0000 7f24eab83700 0 -- 172.21.15.33:0/1123468221 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f24cc02ddb0 msgr2=0x7f24cc0301d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:30:02.158 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.158+0000 7f24fd959ec0 20 add_watcher() i=0 2019-12-09T09:30:02.158 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.158+0000 7f24fd959ec0 20 add_watcher() i=1 2019-12-09T09:30:02.159 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.158+0000 7f24fd959ec0 20 add_watcher() i=2 2019-12-09T09:30:02.159 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.158+0000 7f24fd959ec0 20 add_watcher() i=3 2019-12-09T09:30:02.159 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.158+0000 7f24fd959ec0 20 add_watcher() i=4 2019-12-09T09:30:02.159 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.158+0000 7f24fd959ec0 20 add_watcher() i=5 2019-12-09T09:30:02.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.158+0000 7f24fd959ec0 20 add_watcher() i=6 2019-12-09T09:30:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.207+0000 7f24fd959ec0 20 add_watcher() i=7 2019-12-09T09:30:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.207+0000 7f24fd959ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:30:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:02.207+0000 7f24bb7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:30:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:06.615+0000 7f24fd959ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2019-12-09T09:30:11.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:11.256+0000 7f24fd959ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.log 2019-12-09T09:30:15.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.376+0000 7f2421ffb700 20 reqs_thread_entry: start 2019-12-09T09:30:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.376+0000 7f24137fe700 20 reqs_thread_entry: start 2019-12-09T09:30:15.376 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.376+0000 7f24fd959ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:30:15.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.377+0000 7f2411ffb700 20 reqs_thread_entry: start 2019-12-09T09:30:15.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.622+0000 7f24fd959ec0 20 remove_watcher() i=0 2019-12-09T09:30:15.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.622+0000 7f24fd959ec0 2 removed watcher, disabling cache 2019-12-09T09:30:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.704+0000 7f24fd959ec0 20 remove_watcher() i=1 2019-12-09T09:30:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.782+0000 7f24fd959ec0 20 remove_watcher() i=2 2019-12-09T09:30:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.848+0000 7f24fd959ec0 20 remove_watcher() i=3 2019-12-09T09:30:15.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.888+0000 7f24fd959ec0 20 remove_watcher() i=4 2019-12-09T09:30:15.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:15.926+0000 7f24fd959ec0 20 remove_watcher() i=5 2019-12-09T09:30:16.013 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.014+0000 7f24fd959ec0 20 remove_watcher() i=6 2019-12-09T09:30:16.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.076+0000 7f24fd959ec0 20 remove_watcher() i=7 2019-12-09T09:30:16.089 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2019-12-09T09:30:16.089 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T09:30:16.090 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T09:30:16.090 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T09:30:16.090 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2019-12-09T09:30:16.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.314+0000 7f8801ffb700 20 reqs_thread_entry: start 2019-12-09T09:30:16.314 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.314+0000 7f88cccecec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.338+0000 7f88cccecec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:16.337 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.338+0000 7f88cccecec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.351+0000 7f88cccecec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T09:30:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.351+0000 7f88cccecec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.370+0000 7f88cccecec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:16.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.370+0000 7f88cccecec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.387+0000 7f88cccecec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.387+0000 7f88cccecec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.407 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.407+0000 7f88cccecec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:30:16.578 INFO:teuthology.orchestra.run.smithi033.stdout:{"id":"d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:30:16.580 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'd7b54479-3ea8-4d4d-b4ef-b3b6c4677caf', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:30:16.580 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T09:30:16.580 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T09:30:16.580 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T09:30:16.741 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.740+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.769+0000 7fabc0fb0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:16.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.769+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.778+0000 7fabc0fb0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:16.778 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.778+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.786 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.788+0000 7fabc0fb0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:16.787 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.788+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:16.804+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.124+0000 7fabc0fb0ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:30:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.124+0000 7fabc0fb0ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:30:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.124+0000 7fabc0fb0ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:30:17.124 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.124+0000 7fabc0fb0ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:30:17.125 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.124+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.143+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:17.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.143+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.161+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:30:17.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.161+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.181+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:17.180 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.181+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.197+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:30:17.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.197+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.203+0000 7fabc0fb0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.203+0000 7fabc0fb0ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:30:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.203+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.212+0000 7fabc0fb0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:17.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.212+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.225+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:17.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.225+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.237+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:30:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.237+0000 7fabc0fb0ec0 20 zonegroup default 2019-12-09T09:30:17.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.237+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.244+0000 7fabc0fb0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:17.243 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.244+0000 7fabc0fb0ec0 10 Cannot find current period zone using local zone 2019-12-09T09:30:17.244 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.244+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.251 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.253+0000 7fabc0fb0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:17.252 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.253+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.268+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.268+0000 7fabc0fb0ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.288+0000 7fabc0fb0ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:30:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.288+0000 7fabc0fb0ec0 20 zone default found 2019-12-09T09:30:17.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.288+0000 7fabc0fb0ec0 20 started sync module instance, tier type = 2019-12-09T09:30:17.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.288+0000 7fabc0fb0ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:30:17.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.744+0000 7fabc0fb0ec0 20 add_watcher() i=0 2019-12-09T09:30:17.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.744+0000 7fabc0fb0ec0 20 add_watcher() i=1 2019-12-09T09:30:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.744+0000 7fabc0fb0ec0 20 add_watcher() i=2 2019-12-09T09:30:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.744+0000 7fabc0fb0ec0 20 add_watcher() i=3 2019-12-09T09:30:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.744+0000 7fabc0fb0ec0 20 add_watcher() i=4 2019-12-09T09:30:17.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.744+0000 7fabc0fb0ec0 20 add_watcher() i=5 2019-12-09T09:30:17.757 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.758+0000 7fabc0fb0ec0 20 add_watcher() i=6 2019-12-09T09:30:17.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.781+0000 7fabc0fb0ec0 20 add_watcher() i=7 2019-12-09T09:30:17.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.781+0000 7fabc0fb0ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:30:17.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.781+0000 7fab7efdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:30:17.788 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.788+0000 7faba7fff700 0 -- 172.21.15.33:0/3717406303 >> v1:172.21.15.33:6789/0 conn(0x55b1c2cd6fa0 legacy=0x55b1c2cd94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:30:17.830 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:17.831+0000 7faba7fff700 0 -- 172.21.15.33:0/88790611 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x55b1c2cdf8a0 msgr2=0x55b1c2ce1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:30:20.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.514+0000 7faaed7fa700 20 reqs_thread_entry: start 2019-12-09T09:30:20.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.514+0000 7faae6ffd700 20 reqs_thread_entry: start 2019-12-09T09:30:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.514+0000 7fabc0fb0ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:30:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.515+0000 7faae57fa700 20 reqs_thread_entry: start 2019-12-09T09:30:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.716+0000 7fabc0fb0ec0 20 remove_watcher() i=0 2019-12-09T09:30:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.716+0000 7fabc0fb0ec0 2 removed watcher, disabling cache 2019-12-09T09:30:20.752 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.754+0000 7fabc0fb0ec0 20 remove_watcher() i=1 2019-12-09T09:30:20.787 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.789+0000 7fabc0fb0ec0 20 remove_watcher() i=2 2019-12-09T09:30:20.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.856+0000 7fabc0fb0ec0 20 remove_watcher() i=3 2019-12-09T09:30:20.919 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.921+0000 7fabc0fb0ec0 20 remove_watcher() i=4 2019-12-09T09:30:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:20.978+0000 7fabc0fb0ec0 20 remove_watcher() i=5 2019-12-09T09:30:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.050+0000 7fabc0fb0ec0 20 remove_watcher() i=6 2019-12-09T09:30:21.118 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.119+0000 7fabc0fb0ec0 20 remove_watcher() i=7 2019-12-09T09:30:21.130 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2019-12-09T09:30:21.131 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T09:30:21.131 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2019-12-09T09:30:21.131 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T09:30:21.131 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T09:30:21.131 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2019-12-09T09:30:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.340+0000 7fab59ffb700 20 reqs_thread_entry: start 2019-12-09T09:30:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.340+0000 7fac245ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.348+0000 7fac245ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.348+0000 7fac245ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.362+0000 7fac245ebec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.362+0000 7fac245ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.372+0000 7fac245ebec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T09:30:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.372+0000 7fac245ebec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.384+0000 7fac245ebec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:30:21.490 INFO:teuthology.orchestra.run.smithi033.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2019-12-09T09:30:21.490 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T09:30:21.491 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T09:30:21.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T09:30:21.491 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2019-12-09T09:30:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.698+0000 7f79a0ff9700 20 reqs_thread_entry: start 2019-12-09T09:30:21.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.698+0000 7f7a53668ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.726+0000 7f7a53668ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:21.730 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.726+0000 7f7a53668ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.739+0000 7f7a53668ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T09:30:21.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.739+0000 7f7a53668ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.757+0000 7f7a53668ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:21.755 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.757+0000 7f7a53668ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.773 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.775+0000 7f7a53668ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:21.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.775+0000 7f7a53668ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:21.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:21.806+0000 7f7a53668ec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T09:30:21.902 INFO:teuthology.orchestra.run.smithi033.stdout:{"id":"d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:30:21.902 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'd7b54479-3ea8-4d4d-b4ef-b3b6c4677caf', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:30:21.902 INFO:tasks.rgw:Configuring storage class type = FROZEN 2019-12-09T09:30:21.902 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T09:30:21.904 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T09:30:21.904 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2019-12-09T09:30:22.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.117+0000 7ffa47fff700 20 reqs_thread_entry: start 2019-12-09T09:30:22.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.117+0000 7ffb13566ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.134+0000 7ffb13566ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.134+0000 7ffb13566ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.162+0000 7ffb13566ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:22.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.162+0000 7ffb13566ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.188+0000 7ffb13566ec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T09:30:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.188+0000 7ffb13566ec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.238+0000 7ffb13566ec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T09:30:22.329 INFO:teuthology.orchestra.run.smithi033.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2019-12-09T09:30:22.330 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'FROZEN', u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T09:30:22.330 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T09:30:22.331 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T09:30:22.331 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2019-12-09T09:30:22.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.567+0000 7f4e6bfff700 20 reqs_thread_entry: start 2019-12-09T09:30:22.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.567+0000 7f4f374fbec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.590+0000 7f4f374fbec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:22.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.590+0000 7f4f374fbec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.602+0000 7f4f374fbec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T09:30:22.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.603+0000 7f4f374fbec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.615+0000 7f4f374fbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:30:22.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.615+0000 7f4f374fbec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.627+0000 7f4f374fbec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:30:22.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.627+0000 7f4f374fbec0 20 rados->read ofs=0 len=0 2019-12-09T09:30:22.645 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:30:22.646+0000 7f4f374fbec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:30:22.850 INFO:teuthology.orchestra.run.smithi033.stdout:{"id":"d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T09:30:22.851 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'FROZEN': {u'data_pool': u'default.rgw.buckets.data.frozen'}, u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'd7b54479-3ea8-4d4d-b4ef-b3b6c4677caf', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T09:30:22.851 INFO:tasks.rgw:Starting rgw... 2019-12-09T09:30:22.851 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2019-12-09T09:30:22.851 INFO:tasks.rgw:Using civetweb as radosgw frontend 2019-12-09T09:30:22.851 DEBUG:teuthology.misc:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--tool=memcheck', '--max-threads=1024'] 2019-12-09T09:30:22.852 INFO:tasks.rgw.client.0:Restarting daemon 2019-12-09T09:30:22.852 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2019-12-09T09:30:22.856 INFO:tasks.rgw.client.0:Started 2019-12-09T09:30:22.857 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi033.front.sepia.ceph.com:80/ 2019-12-09T09:30:22.857 INFO:teuthology.orchestra.run.smithi033:> curl http://smithi033.front.sepia.ceph.com:80/ 2019-12-09T09:30:22.943 INFO:teuthology.orchestra.run.smithi033.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:30:22.945 INFO:teuthology.orchestra.run.smithi033.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:30:22.948 INFO:teuthology.orchestra.run.smithi033.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi033.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:30:22.948 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:30:23.961 INFO:teuthology.orchestra.run.smithi033:> curl http://smithi033.front.sepia.ceph.com:80/ 2019-12-09T09:30:24.010 INFO:teuthology.orchestra.run.smithi033.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:30:24.010 INFO:teuthology.orchestra.run.smithi033.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:30:24.016 INFO:teuthology.orchestra.run.smithi033.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi033.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:30:24.016 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:30:26.017 INFO:teuthology.orchestra.run.smithi033:> curl http://smithi033.front.sepia.ceph.com:80/ 2019-12-09T09:30:26.102 INFO:teuthology.orchestra.run.smithi033.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:30:26.102 INFO:teuthology.orchestra.run.smithi033.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:30:26.105 INFO:teuthology.orchestra.run.smithi033.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi033.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:30:26.106 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:30:30.107 INFO:teuthology.orchestra.run.smithi033:> curl http://smithi033.front.sepia.ceph.com:80/ 2019-12-09T09:30:30.189 INFO:teuthology.orchestra.run.smithi033.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:30:30.190 INFO:teuthology.orchestra.run.smithi033.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:30:30.195 INFO:teuthology.orchestra.run.smithi033.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi033.front.sepia.ceph.com:80; Connection refused 2019-12-09T09:30:30.195 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T09:30:38.197 INFO:teuthology.orchestra.run.smithi033:> curl http://smithi033.front.sepia.ceph.com:80/ 2019-12-09T09:30:38.267 INFO:teuthology.orchestra.run.smithi033.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T09:30:38.267 INFO:teuthology.orchestra.run.smithi033.stderr: Dload Upload Total Spent Left Speed 2019-12-09T09:30:38.969 INFO:teuthology.orchestra.run.smithi033.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 304 0 --:--:-- --:--:-- --:--:-- 304 2019-12-09T09:30:38.969 INFO:teuthology.orchestra.run.smithi033.stdout:anonymous 2019-12-09T09:30:38.970 INFO:teuthology.run_tasks:Running task workunit... 2019-12-09T09:30:39.037 INFO:tasks.workunit:Pulling workunits from ref 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:30:39.038 INFO:tasks.workunit:Making a separate scratch dir for every client... 2019-12-09T09:30:39.038 INFO:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:30:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:stat: cannot stat ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2019-12-09T09:30:39.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T09:30:39.113 INFO:teuthology.orchestra.run.smithi033:> mkdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:30:39.179 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2019-12-09T09:30:39.179 INFO:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2019-12-09T09:30:39.230 INFO:tasks.workunit:timeout=3h 2019-12-09T09:30:39.231 INFO:tasks.workunit:cleanup=True 2019-12-09T09:30:39.231 INFO:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T09:30:39.567 INFO:tasks.workunit.client.0.smithi033.stdout:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2019-12-09T09:31:45.864 INFO:tasks.workunit.client.0.smithi033.stderr:Note: checking out '1351c819e1a9518cde4c8b12d645782009028360'. 2019-12-09T09:31:45.866 INFO:tasks.workunit.client.0.smithi033.stderr: 2019-12-09T09:31:45.866 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2019-12-09T09:31:45.866 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2019-12-09T09:31:45.867 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by performing another checkout. 2019-12-09T09:31:45.867 INFO:tasks.workunit.client.0.smithi033.stderr: 2019-12-09T09:31:45.867 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2019-12-09T09:31:45.867 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -b with the checkout command again. Example: 2019-12-09T09:31:45.867 INFO:tasks.workunit.client.0.smithi033.stderr: 2019-12-09T09:31:45.868 INFO:tasks.workunit.client.0.smithi033.stderr: git checkout -b new_branch_name 2019-12-09T09:31:45.868 INFO:tasks.workunit.client.0.smithi033.stderr: 2019-12-09T09:31:45.868 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 1351c81... rgw: Fix dynamic resharding not working for empty zonegroup in period 2019-12-09T09:31:45.870 INFO:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2019-12-09T09:31:45.924 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2019-12-09T09:31:45.924 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T09:31:45.925 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2019-12-09T09:31:46.439 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2019-12-09T09:31:46.527 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2019-12-09T09:31:46.564 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T09:31:46.566 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T09:31:46.566 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2019-12-09T09:31:46.606 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T09:31:46.659 DEBUG:teuthology.orchestra.remote:smithi033:/home/ubuntu/cephtest/workunits.list.client.0 is 4KB 2019-12-09T09:31:46.705 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2019-12-09T09:31:46.706 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2019-12-09T09:31:46.706 INFO:teuthology.orchestra.run.smithi033:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2019-12-09T09:31:46.761 INFO:tasks.workunit.client.0.smithi033.stdout:Running main() from gmock_main.cc 2019-12-09T09:31:46.761 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] Running 11 tests from 1 test suite. 2019-12-09T09:31:46.762 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment set-up. 2019-12-09T09:31:46.762 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 11 tests from ClsLock 2019-12-09T09:31:46.762 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestMultiLocking 2019-12-09T09:31:49.824 INFO:tasks.workunit.client.0.smithi033.stdout:lock: mylock 2019-12-09T09:31:49.824 INFO:tasks.workunit.client.0.smithi033.stdout: lock_type: exclusive 2019-12-09T09:31:49.824 INFO:tasks.workunit.client.0.smithi033.stdout: tag: 2019-12-09T09:31:49.824 INFO:tasks.workunit.client.0.smithi033.stdout: lockers: 2019-12-09T09:31:49.824 INFO:tasks.workunit.client.0.smithi033.stdout: client.4483 expiration=2019-12-09T09:33:49.696674+0000 addr=v1:172.21.15.33:0/1238820840 cookie= 2019-12-09T09:31:49.987 INFO:tasks.workunit.client.0.smithi033.stdout:lock: mylock 2019-12-09T09:31:49.988 INFO:tasks.workunit.client.0.smithi033.stdout: lock_type: shared 2019-12-09T09:31:49.988 INFO:tasks.workunit.client.0.smithi033.stdout: tag: 2019-12-09T09:31:49.988 INFO:tasks.workunit.client.0.smithi033.stdout: lockers: 2019-12-09T09:31:49.988 INFO:tasks.workunit.client.0.smithi033.stdout: client.4483 expiration=2019-12-09T09:33:49.923783+0000 addr=v1:172.21.15.33:0/1238820840 cookie= 2019-12-09T09:31:49.988 INFO:tasks.workunit.client.0.smithi033.stdout: client.4551 expiration=0.000000 addr=v1:172.21.15.33:0/2165792858 cookie= 2019-12-09T09:31:50.040 INFO:tasks.workunit.client.0.smithi033.stdout:lock: mylock 2019-12-09T09:31:50.040 INFO:tasks.workunit.client.0.smithi033.stdout: lock_type: shared 2019-12-09T09:31:50.040 INFO:tasks.workunit.client.0.smithi033.stdout: tag: 2019-12-09T09:31:50.040 INFO:tasks.workunit.client.0.smithi033.stdout: lockers: 2019-12-09T09:31:50.041 INFO:tasks.workunit.client.0.smithi033.stdout: client.4551 expiration=0.000000 addr=v1:172.21.15.33:0/2165792858 cookie= 2019-12-09T09:31:51.406 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestMultiLocking (4645 ms) 2019-12-09T09:31:51.406 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestMeta 2019-12-09T09:31:54.223 INFO:tasks.workunit.client.0.smithi033.stdout:lock: mylock 2019-12-09T09:31:54.224 INFO:tasks.workunit.client.0.smithi033.stdout: lock_type: shared 2019-12-09T09:31:54.224 INFO:tasks.workunit.client.0.smithi033.stdout: tag: 2019-12-09T09:31:54.224 INFO:tasks.workunit.client.0.smithi033.stdout: lockers: 2019-12-09T09:31:54.224 INFO:tasks.workunit.client.0.smithi033.stdout: client.4563 expiration=0.000000 addr=v1:172.21.15.33:0/1176350657 cookie= 2019-12-09T09:31:54.326 INFO:tasks.workunit.client.0.smithi033.stdout:lock: mylock 2019-12-09T09:31:54.326 INFO:tasks.workunit.client.0.smithi033.stdout: lock_type: exclusive 2019-12-09T09:31:54.326 INFO:tasks.workunit.client.0.smithi033.stdout: tag: new_tag 2019-12-09T09:31:54.326 INFO:tasks.workunit.client.0.smithi033.stdout: lockers: 2019-12-09T09:31:54.327 INFO:tasks.workunit.client.0.smithi033.stdout: client.4492 expiration=0.000000 addr=v1:172.21.15.33:0/2488131596 cookie= 2019-12-09T09:31:54.955 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestMeta (3550 ms) 2019-12-09T09:31:54.956 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestCookie 2019-12-09T09:31:57.926 INFO:tasks.workunit.client.0.smithi033.stdout:lock: mylock 2019-12-09T09:31:57.926 INFO:tasks.workunit.client.0.smithi033.stdout: lock_type: exclusive 2019-12-09T09:31:57.926 INFO:tasks.workunit.client.0.smithi033.stdout: tag: 2019-12-09T09:31:57.926 INFO:tasks.workunit.client.0.smithi033.stdout: lockers: 2019-12-09T09:31:57.997 INFO:tasks.workunit.client.0.smithi033.stdout:lock: mylock 2019-12-09T09:31:57.997 INFO:tasks.workunit.client.0.smithi033.stdout: lock_type: shared 2019-12-09T09:31:57.998 INFO:tasks.workunit.client.0.smithi033.stdout: tag: 2019-12-09T09:31:57.998 INFO:tasks.workunit.client.0.smithi033.stdout: lockers: 2019-12-09T09:31:57.998 INFO:tasks.workunit.client.0.smithi033.stdout: client.4572 expiration=0.000000 addr=v1:172.21.15.33:0/1348202885 cookie= 2019-12-09T09:31:57.998 INFO:tasks.workunit.client.0.smithi033.stdout: client.4572 expiration=0.000000 addr=v1:172.21.15.33:0/1348202885 cookie=new cookie 2019-12-09T09:31:58.256 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestCookie (3294 ms) 2019-12-09T09:31:58.256 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestMultipleLocks 2019-12-09T09:32:01.653 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestMultipleLocks (3403 ms) 2019-12-09T09:32:01.653 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestLockDuration 2019-12-09T09:32:09.898 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestLockDuration (8245 ms) 2019-12-09T09:32:09.898 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestAssertLocked 2019-12-09T09:32:13.335 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestAssertLocked (3433 ms) 2019-12-09T09:32:13.335 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestSetCookie 2019-12-09T09:32:16.788 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestSetCookie (3458 ms) 2019-12-09T09:32:16.788 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestRenew 2019-12-09T09:32:38.304 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestRenew (21515 ms) 2019-12-09T09:32:38.304 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2019-12-09T09:32:46.007 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7703 ms) 2019-12-09T09:32:46.007 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2019-12-09T09:32:54.142 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8136 ms) 2019-12-09T09:32:54.142 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2019-12-09T09:33:02.312 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8169 ms) 2019-12-09T09:33:02.313 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 11 tests from ClsLock (75551 ms total) 2019-12-09T09:33:02.313 INFO:tasks.workunit.client.0.smithi033.stdout: 2019-12-09T09:33:02.313 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment tear-down 2019-12-09T09:33:02.313 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] 11 tests from 1 test suite ran. (75551 ms total) 2019-12-09T09:33:02.313 INFO:tasks.workunit.client.0.smithi033.stdout:[ PASSED ] 11 tests. 2019-12-09T09:33:02.314 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:33:02.315 INFO:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:33:02.432 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2019-12-09T09:33:02.433 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2019-12-09T09:33:02.433 INFO:teuthology.orchestra.run.smithi033:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2019-12-09T09:33:02.489 INFO:tasks.workunit.client.0.smithi033.stdout:Running main() from gmock_main.cc 2019-12-09T09:33:02.489 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] Running 4 tests from 1 test suite. 2019-12-09T09:33:02.490 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment set-up. 2019-12-09T09:33:02.490 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 4 tests from cls_log 2019-12-09T09:33:02.490 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_log.test_log_add_same_time 2019-12-09T09:33:05.713 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_log.test_log_add_same_time (3224 ms) 2019-12-09T09:33:05.714 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_log.test_log_add_different_time 2019-12-09T09:33:09.345 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_log.test_log_add_different_time (3632 ms) 2019-12-09T09:33:09.345 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_log.trim_by_time 2019-12-09T09:33:14.026 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_log.trim_by_time (4681 ms) 2019-12-09T09:33:14.026 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_log.trim_by_marker 2019-12-09T09:33:18.544 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_log.trim_by_marker (4518 ms) 2019-12-09T09:33:18.544 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 4 tests from cls_log (16055 ms total) 2019-12-09T09:33:18.545 INFO:tasks.workunit.client.0.smithi033.stdout: 2019-12-09T09:33:18.545 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment tear-down 2019-12-09T09:33:18.545 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] 4 tests from 1 test suite ran. (16055 ms total) 2019-12-09T09:33:18.545 INFO:tasks.workunit.client.0.smithi033.stdout:[ PASSED ] 4 tests. 2019-12-09T09:33:18.546 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:33:18.546 INFO:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:33:18.614 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2019-12-09T09:33:18.614 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2019-12-09T09:33:18.614 INFO:teuthology.orchestra.run.smithi033:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2019-12-09T09:33:18.668 INFO:tasks.workunit.client.0.smithi033.stdout:Running main() from gmock_main.cc 2019-12-09T09:33:18.668 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] Running 5 tests from 1 test suite. 2019-12-09T09:33:18.669 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment set-up. 2019-12-09T09:33:18.669 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 5 tests from cls_rgw 2019-12-09T09:33:18.669 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.test_implicit 2019-12-09T09:33:22.076 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.test_implicit (3408 ms) 2019-12-09T09:33:22.076 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2019-12-09T09:33:25.497 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3421 ms) 2019-12-09T09:33:25.497 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.test_put_snap 2019-12-09T09:33:34.259 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T09:33:34.215+0000 33f8b700 -1 osd.5 pg_epoch: 105 pg[25.6( v 104'4 (0'0,104'4] local-lis/les=101/102 n=2 ec=101/101 lis/c=101/101 les/c/f=102/102/0 sis=101) [5,1] r=0 lpr=101 crt=103'3 lcod 103'3 mlcod 103'3 active+clean+snaptrim trimq=[1~1]] removing snap head 2019-12-09T09:33:35.453 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.test_put_snap (9952 ms) 2019-12-09T09:33:35.453 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.test_explicit 2019-12-09T09:33:39.028 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.test_explicit (3579 ms) 2019-12-09T09:33:39.028 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.set 2019-12-09T09:33:42.382 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.set (3353 ms) 2019-12-09T09:33:42.383 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 5 tests from cls_rgw (23713 ms total) 2019-12-09T09:33:42.383 INFO:tasks.workunit.client.0.smithi033.stdout: 2019-12-09T09:33:42.383 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment tear-down 2019-12-09T09:33:42.383 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] 5 tests from 1 test suite ran. (23713 ms total) 2019-12-09T09:33:42.383 INFO:tasks.workunit.client.0.smithi033.stdout:[ PASSED ] 5 tests. 2019-12-09T09:33:42.384 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:33:42.385 INFO:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:33:42.506 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2019-12-09T09:33:42.507 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2019-12-09T09:33:42.507 INFO:teuthology.orchestra.run.smithi033:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2019-12-09T09:33:42.563 INFO:tasks.workunit.client.0.smithi033.stdout:Running main() from gmock_main.cc 2019-12-09T09:33:42.563 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] Running 13 tests from 1 test suite. 2019-12-09T09:33:42.563 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment set-up. 2019-12-09T09:33:42.564 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 13 tests from cls_rgw 2019-12-09T09:33:44.624 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.index_basic 2019-12-09T09:33:46.135 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.index_basic (1512 ms) 2019-12-09T09:33:46.135 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2019-12-09T09:33:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (1362 ms) 2019-12-09T09:33:47.511 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.index_remove_object 2019-12-09T09:33:49.573 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.index_remove_object (2076 ms) 2019-12-09T09:33:49.574 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.index_suggest 2019-12-09T09:34:16.946 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.index_suggest (27372 ms) 2019-12-09T09:34:16.948 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.index_list 2019-12-09T09:34:18.506 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.index_list (1561 ms) 2019-12-09T09:34:18.506 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.bi_list 2019-12-09T09:34:21.566 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.bi_list (3060 ms) 2019-12-09T09:34:21.566 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.gc_set 2019-12-09T09:34:22.173 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.gc_set (607 ms) 2019-12-09T09:34:22.174 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.gc_list 2019-12-09T09:34:22.758 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.gc_list (585 ms) 2019-12-09T09:34:22.759 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.gc_defer 2019-12-09T09:34:31.367 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.gc_defer (8609 ms) 2019-12-09T09:34:31.367 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.usage_basic 2019-12-09T09:34:38.303 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.usage_basic (6936 ms) 2019-12-09T09:34:38.303 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2019-12-09T09:34:38.331 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.usage_clear_no_obj (28 ms) 2019-12-09T09:34:38.332 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.usage_clear 2019-12-09T09:34:48.131 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.usage_clear (9800 ms) 2019-12-09T09:34:48.132 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] cls_rgw.bi_log_trim 2019-12-09T09:34:50.168 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] cls_rgw.bi_log_trim (2037 ms) 2019-12-09T09:34:51.363 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 13 tests from cls_rgw (65545 ms total) 2019-12-09T09:34:51.363 INFO:tasks.workunit.client.0.smithi033.stdout: 2019-12-09T09:34:51.363 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment tear-down 2019-12-09T09:34:51.363 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] 13 tests from 1 test suite ran. (68801 ms total) 2019-12-09T09:34:51.364 INFO:tasks.workunit.client.0.smithi033.stdout:[ PASSED ] 13 tests. 2019-12-09T09:34:51.365 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:34:51.365 INFO:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:34:51.500 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2019-12-09T09:34:51.501 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2019-12-09T09:34:51.501 INFO:teuthology.orchestra.run.smithi033:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2019-12-09T09:34:51.596 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] Running 6 tests from 1 test suite. 2019-12-09T09:34:51.597 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment set-up. 2019-12-09T09:34:53.594 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 6 tests from rgw_gc_log 2019-12-09T09:34:53.594 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2019-12-09T09:34:54.124 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] rgw_gc_log.init_existing_queue (511 ms) 2019-12-09T09:34:54.124 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2019-12-09T09:34:54.396 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] rgw_gc_log.init_existing_omap (291 ms) 2019-12-09T09:34:54.396 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2019-12-09T09:34:54.490 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (91 ms) 2019-12-09T09:34:54.490 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2019-12-09T09:34:54.552 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (65 ms) 2019-12-09T09:34:54.553 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2019-12-09T09:34:54.847 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] rgw_gc_log.defer1_after_init (294 ms) 2019-12-09T09:34:54.847 INFO:tasks.workunit.client.0.smithi033.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2019-12-09T09:34:54.903 INFO:tasks.workunit.client.0.smithi033.stdout:[ OK ] rgw_gc_log.defer2_before_init (56 ms) 2019-12-09T09:34:54.903 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] 6 tests from rgw_gc_log (1309 ms total) 2019-12-09T09:34:54.904 INFO:tasks.workunit.client.0.smithi033.stdout: 2019-12-09T09:34:54.904 INFO:tasks.workunit.client.0.smithi033.stdout:[----------] Global test environment tear-down 2019-12-09T09:34:55.950 INFO:tasks.workunit.client.0.smithi033.stdout:[==========] 6 tests from 1 test suite ran. (4355 ms total) 2019-12-09T09:34:55.951 INFO:tasks.workunit.client.0.smithi033.stdout:[ PASSED ] 6 tests. 2019-12-09T09:34:55.953 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T09:34:55.953 INFO:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T09:34:56.020 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh'] on client.0... 2019-12-09T09:34:56.020 INFO:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2019-12-09T09:34:56.270 DEBUG:teuthology.parallel:result is None 2019-12-09T09:34:56.270 INFO:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:34:56.333 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:34:56.333 INFO:teuthology.orchestra.run.smithi033:> rmdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T09:34:56.405 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T09:34:56.405 INFO:teuthology.run_tasks:Running task ragweed... 2019-12-09T09:34:56.418 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}} 2019-12-09T09:34:56.418 INFO:tasks.ragweed:Downloading ragweed... 2019-12-09T09:34:56.419 INFO:tasks.ragweed:Using branch 'wip-ofriedma-dynamic-disable' for ragweed 2019-12-09T09:34:56.419 INFO:teuthology.orchestra.run.smithi033:> git clone -b wip-ofriedma-dynamic-disable git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T09:34:56.476 INFO:teuthology.orchestra.run.smithi033.stderr:fatal: Remote branch wip-ofriedma-dynamic-disable not found in upstream origin 2019-12-09T09:34:56.477 INFO:teuthology.orchestra.run.smithi033.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T09:34:56.477 DEBUG:teuthology.orchestra.run:got remote process result: 128 2019-12-09T09:34:56.478 INFO:teuthology.orchestra.run.smithi033:> git clone -b ceph-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T09:34:56.568 INFO:teuthology.orchestra.run.smithi033.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T09:34:56.670 INFO:tasks.ragweed:Creating rgw users... 2019-12-09T09:34:56.671 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2019-12-09T09:34:56.671 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key LZXFPEAOSHYKKAQLAIRH --secret UGNRqV2BjvZ+72J8YUWImhxcHdv/syfyfpXWhhCV7XwxaQBoWzRrFw== --email sysuser.client.0+test@test.test --system true 2019-12-09T09:34:56.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.806+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:56.920 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.923+0000 7fcdff9d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:34:56.920 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.923+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:56.930 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.934+0000 7fcdff9d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:34:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.934+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:56.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.941+0000 7fcdff9d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:34:56.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.941+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:56.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:56.956+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:34:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.210+0000 7fcdff9d6ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:34:57.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.210+0000 7fcdff9d6ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:34:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.210+0000 7fcdff9d6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:34:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.210+0000 7fcdff9d6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:34:57.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.210+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.222+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:34:57.219 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.222+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.230+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:34:57.227 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.230+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.241+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:34:57.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.241+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.256+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:34:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.256+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.261+0000 7fcdff9d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:34:57.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.261+0000 7fcdff9d6ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:34:57.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.261+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.270+0000 7fcdff9d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:34:57.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.270+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.281+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:34:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.281+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.320 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.324+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:34:57.321 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.324+0000 7fcdff9d6ec0 20 zonegroup default 2019-12-09T09:34:57.321 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.324+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.330 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.334+0000 7fcdff9d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:34:57.330 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.334+0000 7fcdff9d6ec0 10 Cannot find current period zone using local zone 2019-12-09T09:34:57.331 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.334+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.336 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.339+0000 7fcdff9d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:34:57.336 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.339+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.348+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:34:57.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.348+0000 7fcdff9d6ec0 20 rados->read ofs=0 len=0 2019-12-09T09:34:57.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.358+0000 7fcdff9d6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:34:57.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.358+0000 7fcdff9d6ec0 20 zone default found 2019-12-09T09:34:57.356 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.358+0000 7fcdff9d6ec0 20 started sync module instance, tier type = 2019-12-09T09:34:57.356 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.358+0000 7fcdff9d6ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:34:57.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.486+0000 7fcdecc00700 0 -- 172.21.15.33:0/1850288436 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x5627eba29b30 msgr2=0x5627eba2bf90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:34:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.662+0000 7fcdff9d6ec0 20 add_watcher() i=0 2019-12-09T09:34:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.662+0000 7fcdff9d6ec0 20 add_watcher() i=1 2019-12-09T09:34:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.662+0000 7fcdff9d6ec0 20 add_watcher() i=2 2019-12-09T09:34:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.662+0000 7fcdff9d6ec0 20 add_watcher() i=3 2019-12-09T09:34:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.662+0000 7fcdff9d6ec0 20 add_watcher() i=4 2019-12-09T09:34:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.662+0000 7fcdff9d6ec0 20 add_watcher() i=5 2019-12-09T09:34:57.668 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.671+0000 7fcdff9d6ec0 20 add_watcher() i=6 2019-12-09T09:34:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.695+0000 7fcdff9d6ec0 20 add_watcher() i=7 2019-12-09T09:34:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.695+0000 7fcdff9d6ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:34:57.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.695+0000 7fcdbcfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:34:57.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:57.707+0000 7fcdecc00700 0 -- 172.21.15.33:0/556304798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5627eba3bfa0 msgr2=0x5627eba42680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:34:59.230 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.234+0000 7fcd2effd700 20 reqs_thread_entry: start 2019-12-09T09:34:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.234+0000 7fcd2cff9700 20 reqs_thread_entry: start 2019-12-09T09:34:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.234+0000 7fcdff9d6ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:34:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.235+0000 7fcd22ffd700 20 reqs_thread_entry: start 2019-12-09T09:34:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.235+0000 7fcdff9d6ec0 20 get_system_obj_state: rctx=0x7ffea70b1f68 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x5627ebabd800 s->prefetch_data=0 2019-12-09T09:34:59.232 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.235+0000 7fcdff9d6ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T09:34:59.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.260+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2019-12-09T09:34:59.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.260+0000 7fcdff9d6ec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:34:59.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.260+0000 7fcdff9d6ec0 20 get_system_obj_state: rctx=0x7ffea70b1f78 obj=default.rgw.meta:users.email:sysuser.client.0+test@test.test state=0x5627ebabf590 s->prefetch_data=0 2019-12-09T09:34:59.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.260+0000 7fcdff9d6ec0 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2019-12-09T09:34:59.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.268+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2019-12-09T09:34:59.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.268+0000 7fcdff9d6ec0 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:34:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.268+0000 7fcdff9d6ec0 20 get_system_obj_state: rctx=0x7ffea70b1f78 obj=default.rgw.meta:users.keys:LZXFPEAOSHYKKAQLAIRH state=0x5627ebabf590 s->prefetch_data=0 2019-12-09T09:34:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.268+0000 7fcdff9d6ec0 10 cache get: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH : miss 2019-12-09T09:34:59.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.282+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH info.flags=0x0 2019-12-09T09:34:59.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.282+0000 7fcdff9d6ec0 10 adding default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH to cache LRU end 2019-12-09T09:34:59.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.282+0000 7fcdff9d6ec0 20 get_system_obj_state: rctx=0x7ffea70b1b88 obj=default.rgw.meta:users.keys:LZXFPEAOSHYKKAQLAIRH state=0x5627ebabf590 s->prefetch_data=0 2019-12-09T09:34:59.280 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.282+0000 7fcdff9d6ec0 10 cache get: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH : type miss (requested=0x6, cached=0x0) 2019-12-09T09:34:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.291+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH info.flags=0x0 2019-12-09T09:34:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.291+0000 7fcdff9d6ec0 10 moving default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH to cache LRU end 2019-12-09T09:34:59.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.291+0000 7fcdff9d6ec0 20 get_system_obj_state: rctx=0x7ffea70b1d08 obj=default.rgw.meta:users.keys:LZXFPEAOSHYKKAQLAIRH state=0x5627ebabf590 s->prefetch_data=0 2019-12-09T09:34:59.289 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.291+0000 7fcdff9d6ec0 10 cache get: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH : type miss (requested=0x6, cached=0x0) 2019-12-09T09:34:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.305+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH info.flags=0x0 2019-12-09T09:34:59.301 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.305+0000 7fcdff9d6ec0 10 moving default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH to cache LRU end 2019-12-09T09:34:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.466+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T09:34:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.466+0000 7fcdff9d6ec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:34:59.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.466+0000 7fcdff9d6ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=637 2019-12-09T09:34:59.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.512+0000 7fcdd5ffb700 10 RGWWatcher::handle_notify() notify_id 541165879296 cookie 94729457037360 notifier 4692 bl.length()=637 2019-12-09T09:34:59.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.512+0000 7fcdd5ffb700 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T09:34:59.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.512+0000 7fcdd5ffb700 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:34:59.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.855+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T09:34:59.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.855+0000 7fcdff9d6ec0 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:34:59.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.855+0000 7fcdff9d6ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=202 2019-12-09T09:34:59.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.870+0000 7fcdd5ffb700 10 RGWWatcher::handle_notify() notify_id 541165879297 cookie 94729457041872 notifier 4692 bl.length()=202 2019-12-09T09:34:59.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.870+0000 7fcdd5ffb700 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T09:34:59.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:34:59.870+0000 7fcdd5ffb700 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T09:35:00.475 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.478+0000 7fcdff9d6ec0 10 cache put: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH info.flags=0x7 2019-12-09T09:35:00.475 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.478+0000 7fcdff9d6ec0 10 moving default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH to cache LRU end 2019-12-09T09:35:00.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.478+0000 7fcdff9d6ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=190 2019-12-09T09:35:00.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.517+0000 7fcdd5ffb700 10 RGWWatcher::handle_notify() notify_id 541165879296 cookie 94729457018496 notifier 4692 bl.length()=190 2019-12-09T09:35:00.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.517+0000 7fcdd5ffb700 10 cache put: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH info.flags=0x7 2019-12-09T09:35:00.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.517+0000 7fcdd5ffb700 10 moving default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH to cache LRU end 2019-12-09T09:35:00.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.578+0000 7fcdd5ffb700 10 RGWWatcher::handle_notify() notify_id 541165879297 cookie 94729457018496 notifier 4692 bl.length()=190 2019-12-09T09:35:00.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.578+0000 7fcdd5ffb700 10 cache put: name=default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH info.flags=0x7 2019-12-09T09:35:00.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.578+0000 7fcdd5ffb700 10 moving default.rgw.meta+users.keys+LZXFPEAOSHYKKAQLAIRH to cache LRU end 2019-12-09T09:35:00.641 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T09:35:00.641 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "sysuser.client.0", 2019-12-09T09:35:00.642 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Mr. sysuser.client.0", 2019-12-09T09:35:00.642 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "sysuser.client.0+test@test.test", 2019-12-09T09:35:00.642 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T09:35:00.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T09:35:00.642 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [], 2019-12-09T09:35:00.642 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [ 2019-12-09T09:35:00.642 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T09:35:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "sysuser.client.0", 2019-12-09T09:35:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "access_key": "LZXFPEAOSHYKKAQLAIRH", 2019-12-09T09:35:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "UGNRqV2BjvZ+72J8YUWImhxcHdv/syfyfpXWhhCV7XwxaQBoWzRrFw==" 2019-12-09T09:35:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T09:35:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T09:35:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [], 2019-12-09T09:35:00.643 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T09:35:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T09:35:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "system": "true", 2019-12-09T09:35:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T09:35:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T09:35:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T09:35:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T09:35:00.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T09:35:00.645 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T09:35:00.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T09:35:00.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T09:35:00.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T09:35:00.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T09:35:00.646 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T09:35:00.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T09:35:00.646 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T09:35:00.647 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T09:35:00.647 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.695+0000 7fcdff9d6ec0 20 remove_watcher() i=0 2019-12-09T09:35:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.695+0000 7fcdff9d6ec0 2 removed watcher, disabling cache 2019-12-09T09:35:00.763 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.764+0000 7fcdff9d6ec0 20 remove_watcher() i=1 2019-12-09T09:35:00.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.826+0000 7fcdff9d6ec0 20 remove_watcher() i=2 2019-12-09T09:35:00.902 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.905+0000 7fcdff9d6ec0 20 remove_watcher() i=3 2019-12-09T09:35:00.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:00.969+0000 7fcdff9d6ec0 20 remove_watcher() i=4 2019-12-09T09:35:01.023 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.026+0000 7fcdff9d6ec0 20 remove_watcher() i=5 2019-12-09T09:35:01.092 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.096+0000 7fcdff9d6ec0 20 remove_watcher() i=6 2019-12-09T09:35:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.138+0000 7fcdff9d6ec0 20 remove_watcher() i=7 2019-12-09T09:35:01.143 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2019-12-09T09:35:01.144 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key KWPORNRXZWIKYLCKKRHM --secret X0cgxieBlKfd5ZC1VExUJMMHcwxhWGCE8v8y/0o7+vZElNzmEN7JhA== --email ragweed.client.0+test@test.test --system false 2019-12-09T09:35:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.298+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.532 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.535+0000 7ff00b160ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:35:01.532 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.535+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.546+0000 7ff00b160ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:35:01.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.546+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.558+0000 7ff00b160ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:35:01.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.558+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.584+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:35:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.817+0000 7ff00b160ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:35:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.817+0000 7ff00b160ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:35:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.817+0000 7ff00b160ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:35:01.814 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.817+0000 7ff00b160ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:35:01.814 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.817+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.829+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:35:01.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.829+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.840+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.841+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.858+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:35:01.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.858+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.872 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.876+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:35:01.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.876+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.883+0000 7ff00b160ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:35:01.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.883+0000 7ff00b160ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:35:01.883 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.883+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.890+0000 7ff00b160ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:35:01.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.890+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.903+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:35:01.900 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.903+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.916+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:35:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.916+0000 7ff00b160ec0 20 zonegroup default 2019-12-09T09:35:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.916+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.926+0000 7ff00b160ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:35:01.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.926+0000 7ff00b160ec0 10 Cannot find current period zone using local zone 2019-12-09T09:35:01.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.926+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.932+0000 7ff00b160ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:35:01.928 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.932+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.943+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:35:01.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.943+0000 7ff00b160ec0 20 rados->read ofs=0 len=0 2019-12-09T09:35:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.959+0000 7ff00b160ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:35:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.959+0000 7ff00b160ec0 20 zone default found 2019-12-09T09:35:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.959+0000 7ff00b160ec0 20 started sync module instance, tier type = 2019-12-09T09:35:01.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:01.959+0000 7ff00b160ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:35:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.338+0000 7ff00b160ec0 20 add_watcher() i=0 2019-12-09T09:35:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.338+0000 7ff00b160ec0 20 add_watcher() i=1 2019-12-09T09:35:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.338+0000 7ff00b160ec0 20 add_watcher() i=2 2019-12-09T09:35:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.338+0000 7ff00b160ec0 20 add_watcher() i=3 2019-12-09T09:35:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.338+0000 7ff00b160ec0 20 add_watcher() i=4 2019-12-09T09:35:02.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.338+0000 7ff00b160ec0 20 add_watcher() i=5 2019-12-09T09:35:02.336 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.338+0000 7ff00b160ec0 20 add_watcher() i=6 2019-12-09T09:35:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.374+0000 7ff00b160ec0 20 add_watcher() i=7 2019-12-09T09:35:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.374+0000 7ff00b160ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:35:02.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.374+0000 7fefc8fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:35:02.389 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.392+0000 7feff7b89700 0 -- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x555d3486d610 msgr2=0x555d3486fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:35:02.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:02.539+0000 7feff8b8b700 0 -- 172.21.15.33:0/2584390355 >> [v2:172.21.15.186:6810/10132,v1:172.21.15.186:6811/10132] conn(0x555d3487b670 msgr2=0x555d3487da90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:35:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.919+0000 7fef3affd700 20 reqs_thread_entry: start 2019-12-09T09:35:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.919+0000 7fef38ff9700 20 reqs_thread_entry: start 2019-12-09T09:35:03.916 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.919+0000 7ff00b160ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:35:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.920+0000 7fef2effd700 20 reqs_thread_entry: start 2019-12-09T09:35:03.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.920+0000 7ff00b160ec0 20 get_system_obj_state: rctx=0x7fff1cc2dd28 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x555d348df220 s->prefetch_data=0 2019-12-09T09:35:03.920 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.920+0000 7ff00b160ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T09:35:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.933+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2019-12-09T09:35:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.933+0000 7ff00b160ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:35:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.933+0000 7ff00b160ec0 20 get_system_obj_state: rctx=0x7fff1cc2dd38 obj=default.rgw.meta:users.email:ragweed.client.0+test@test.test state=0x555d348e0fb0 s->prefetch_data=0 2019-12-09T09:35:03.930 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.933+0000 7ff00b160ec0 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2019-12-09T09:35:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.948+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2019-12-09T09:35:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.948+0000 7ff00b160ec0 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:35:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.948+0000 7ff00b160ec0 20 get_system_obj_state: rctx=0x7fff1cc2dd38 obj=default.rgw.meta:users.keys:KWPORNRXZWIKYLCKKRHM state=0x555d348e0fb0 s->prefetch_data=0 2019-12-09T09:35:03.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.948+0000 7ff00b160ec0 10 cache get: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM : miss 2019-12-09T09:35:03.955 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.959+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM info.flags=0x0 2019-12-09T09:35:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.959+0000 7ff00b160ec0 10 adding default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM to cache LRU end 2019-12-09T09:35:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.959+0000 7ff00b160ec0 20 get_system_obj_state: rctx=0x7fff1cc2d948 obj=default.rgw.meta:users.keys:KWPORNRXZWIKYLCKKRHM state=0x555d348e0fb0 s->prefetch_data=0 2019-12-09T09:35:03.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.959+0000 7ff00b160ec0 10 cache get: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM : type miss (requested=0x6, cached=0x0) 2019-12-09T09:35:03.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.967+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM info.flags=0x0 2019-12-09T09:35:03.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.967+0000 7ff00b160ec0 10 moving default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM to cache LRU end 2019-12-09T09:35:03.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.967+0000 7ff00b160ec0 20 get_system_obj_state: rctx=0x7fff1cc2dac8 obj=default.rgw.meta:users.keys:KWPORNRXZWIKYLCKKRHM state=0x555d348e0fb0 s->prefetch_data=0 2019-12-09T09:35:03.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.967+0000 7ff00b160ec0 10 cache get: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM : type miss (requested=0x6, cached=0x0) 2019-12-09T09:35:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.977+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM info.flags=0x0 2019-12-09T09:35:03.973 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:03.977+0000 7ff00b160ec0 10 moving default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM to cache LRU end 2019-12-09T09:35:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.064+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T09:35:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.064+0000 7ff00b160ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:35:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.064+0000 7ff00b160ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=637 2019-12-09T09:35:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.079+0000 7fefddffb700 10 RGWWatcher::handle_notify() notify_id 541165879298 cookie 93858801461328 notifier 4481 bl.length()=637 2019-12-09T09:35:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.079+0000 7fefddffb700 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T09:35:04.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.079+0000 7fefddffb700 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:35:04.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.183+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T09:35:04.180 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.183+0000 7ff00b160ec0 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:35:04.180 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.183+0000 7ff00b160ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=202 2019-12-09T09:35:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.194+0000 7fefddffb700 10 RGWWatcher::handle_notify() notify_id 541165879299 cookie 93858801455072 notifier 4481 bl.length()=202 2019-12-09T09:35:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.194+0000 7fefddffb700 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T09:35:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.194+0000 7fefddffb700 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T09:35:04.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.469+0000 7ff00b160ec0 10 cache put: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM info.flags=0x7 2019-12-09T09:35:04.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.469+0000 7ff00b160ec0 10 moving default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM to cache LRU end 2019-12-09T09:35:04.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.469+0000 7ff00b160ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=190 2019-12-09T09:35:04.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.479+0000 7fefddffb700 10 RGWWatcher::handle_notify() notify_id 541165879298 cookie 93858801463904 notifier 4481 bl.length()=190 2019-12-09T09:35:04.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.479+0000 7fefddffb700 10 cache put: name=default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM info.flags=0x7 2019-12-09T09:35:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.479+0000 7fefddffb700 10 moving default.rgw.meta+users.keys+KWPORNRXZWIKYLCKKRHM to cache LRU end 2019-12-09T09:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T09:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "ragweed.client.0", 2019-12-09T09:35:04.706 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Mr. ragweed.client.0", 2019-12-09T09:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "ragweed.client.0+test@test.test", 2019-12-09T09:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T09:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T09:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [], 2019-12-09T09:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [ 2019-12-09T09:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T09:35:04.707 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "ragweed.client.0", 2019-12-09T09:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "access_key": "KWPORNRXZWIKYLCKKRHM", 2019-12-09T09:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "X0cgxieBlKfd5ZC1VExUJMMHcwxhWGCE8v8y/0o7+vZElNzmEN7JhA==" 2019-12-09T09:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T09:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T09:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [], 2019-12-09T09:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T09:35:04.708 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T09:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "system": "true", 2019-12-09T09:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T09:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T09:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T09:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T09:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T09:35:04.709 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T09:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T09:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T09:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T09:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T09:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T09:35:04.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T09:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T09:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T09:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T09:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T09:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T09:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T09:35:04.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T09:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T09:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T09:35:04.712 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.747+0000 7ff00b160ec0 20 remove_watcher() i=0 2019-12-09T09:35:04.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.747+0000 7ff00b160ec0 2 removed watcher, disabling cache 2019-12-09T09:35:04.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.800+0000 7ff00b160ec0 20 remove_watcher() i=1 2019-12-09T09:35:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.854+0000 7ff00b160ec0 20 remove_watcher() i=2 2019-12-09T09:35:04.908 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.912+0000 7ff00b160ec0 20 remove_watcher() i=3 2019-12-09T09:35:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.945+0000 7ff00b160ec0 20 remove_watcher() i=4 2019-12-09T09:35:04.977 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:04.981+0000 7ff00b160ec0 20 remove_watcher() i=5 2019-12-09T09:35:05.018 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:05.021+0000 7ff00b160ec0 20 remove_watcher() i=6 2019-12-09T09:35:05.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:35:05.060+0000 7ff00b160ec0 20 remove_watcher() i=7 2019-12-09T09:35:05.064 INFO:tasks.ragweed:Configuring ragweed... 2019-12-09T09:35:05.064 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest/ragweed && ./bootstrap 2019-12-09T09:35:12.528 INFO:teuthology.orchestra.run.smithi033.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T09:35:12.670 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T09:35:12.710 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T09:35:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T09:35:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T09:35:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T09:35:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T09:35:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T09:35:13.759 INFO:teuthology.orchestra.run.smithi033.stdout:Package python2-pip-8.1.2-10.el7.noarch already installed and latest version 2019-12-09T09:35:14.662 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T09:35:14.663 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:35:14.663 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libevent-devel.x86_64 0:2.0.21-4.el7 will be installed 2019-12-09T09:35:14.665 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 will be installed 2019-12-09T09:35:14.667 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_2.3.x86_64 2019-12-09T09:35:14.710 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libxslt-devel.x86_64 0:1.1.28-5.el7 will be installed 2019-12-09T09:35:14.711 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libgcrypt-devel for package: libxslt-devel-1.1.28-5.el7.x86_64 2019-12-09T09:35:14.717 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed 2019-12-09T09:35:14.717 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:35:14.718 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libgcrypt-devel.x86_64 0:1.5.3-14.el7 will be installed 2019-12-09T09:35:14.718 INFO:teuthology.orchestra.run.smithi033.stdout:--> Processing Dependency: libgpg-error-devel for package: libgcrypt-devel-1.5.3-14.el7.x86_64 2019-12-09T09:35:14.720 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package xz-devel.x86_64 0:5.2.2-1.el7 will be installed 2019-12-09T09:35:14.721 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T09:35:14.721 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libgpg-error-devel.x86_64 0:1.12-3.el7 will be installed 2019-12-09T09:35:15.111 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T09:35:15.313 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:15.313 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T09:35:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:15.314 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:35:15.314 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T09:35:15.315 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:35:15.315 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T09:35:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: libevent-devel x86_64 2.0.21-4.el7 base 85 k 2019-12-09T09:35:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: libxml2-devel x86_64 2.9.1-6.el7_2.3 base 1.0 M 2019-12-09T09:35:15.315 INFO:teuthology.orchestra.run.smithi033.stdout: libxslt-devel x86_64 1.1.28-5.el7 base 309 k 2019-12-09T09:35:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.7-18.el7 base 50 k 2019-12-09T09:35:15.316 INFO:teuthology.orchestra.run.smithi033.stdout:Installing for dependencies: 2019-12-09T09:35:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: libgcrypt-devel x86_64 1.5.3-14.el7 base 129 k 2019-12-09T09:35:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: libgpg-error-devel x86_64 1.12-3.el7 base 16 k 2019-12-09T09:35:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: xz-devel x86_64 5.2.2-1.el7 base 46 k 2019-12-09T09:35:15.316 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:15.317 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T09:35:15.317 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T09:35:15.317 INFO:teuthology.orchestra.run.smithi033.stdout:Install 4 Packages (+3 Dependent packages) 2019-12-09T09:35:15.317 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:15.318 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 1.7 M 2019-12-09T09:35:15.319 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 12 M 2019-12-09T09:35:15.319 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T09:35:15.591 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2019-12-09T09:35:15.591 INFO:teuthology.orchestra.run.smithi033.stdout:Total 6.1 MB/s | 1.7 MB 00:00 2019-12-09T09:35:15.596 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T09:35:15.605 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T09:35:15.662 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T09:35:15.662 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T09:35:16.336 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgpg-error-devel-1.12-3.el7.x86_64 1/7 2019-12-09T09:35:16.657 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgcrypt-devel-1.5.3-14.el7.x86_64 2/7 2019-12-09T09:35:16.928 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T09:35:17.438 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xz-devel-5.2.2-1.el7.x86_64 4/7 2019-12-09T09:35:17.979 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libxml2-devel-2.9.1-6.el7_2.3.x86_64 5/7 2019-12-09T09:35:18.349 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libxslt-devel-1.1.28-5.el7.x86_64 6/7 2019-12-09T09:35:18.632 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libevent-devel-2.0.21-4.el7.x86_64 7/7 2019-12-09T09:35:18.886 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xz-devel-5.2.2-1.el7.x86_64 1/7 2019-12-09T09:35:18.979 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libxml2-devel-2.9.1-6.el7_2.3.x86_64 2/7 2019-12-09T09:35:19.096 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T09:35:19.188 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libxslt-devel-1.1.28-5.el7.x86_64 4/7 2019-12-09T09:35:19.305 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgpg-error-devel-1.12-3.el7.x86_64 5/7 2019-12-09T09:35:19.472 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libevent-devel-2.0.21-4.el7.x86_64 6/7 2019-12-09T09:35:19.973 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgcrypt-devel-1.5.3-14.el7.x86_64 7/7 2019-12-09T09:35:19.973 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:19.973 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T09:35:19.973 INFO:teuthology.orchestra.run.smithi033.stdout: libevent-devel.x86_64 0:2.0.21-4.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 2019-12-09T09:35:19.974 INFO:teuthology.orchestra.run.smithi033.stdout: libxslt-devel.x86_64 0:1.1.28-5.el7 zlib-devel.x86_64 0:1.2.7-18.el7 2019-12-09T09:35:19.974 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:19.974 INFO:teuthology.orchestra.run.smithi033.stdout:Dependency Installed: 2019-12-09T09:35:19.974 INFO:teuthology.orchestra.run.smithi033.stdout: libgcrypt-devel.x86_64 0:1.5.3-14.el7 libgpg-error-devel.x86_64 0:1.12-3.el7 2019-12-09T09:35:19.974 INFO:teuthology.orchestra.run.smithi033.stdout: xz-devel.x86_64 0:5.2.2-1.el7 2019-12-09T09:35:19.974 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:19.974 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T09:35:20.144 INFO:teuthology.orchestra.run.smithi033.stdout:New python executable in /home/ubuntu/cephtest/ragweed/virtualenv/bin/python 2019-12-09T09:35:21.533 INFO:teuthology.orchestra.run.smithi033.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T09:35:22.029 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting pip 2019-12-09T09:35:22.128 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB) 2019-12-09T09:35:22.334 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: pip 2019-12-09T09:35:22.334 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: pip 9.0.1 2019-12-09T09:35:22.405 INFO:teuthology.orchestra.run.smithi033.stdout: Uninstalling pip-9.0.1: 2019-12-09T09:35:22.442 INFO:teuthology.orchestra.run.smithi033.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T09:35:23.045 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed pip-19.3.1 2019-12-09T09:35:23.846 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:35:23.935 INFO:teuthology.orchestra.run.smithi033.stdout:Requirement already satisfied: six in /usr/lib/python2.7/site-packages (1.9.0) 2019-12-09T09:35:24.888 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:35:25.089 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting nose 2019-12-09T09:35:25.142 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB) 2019-12-09T09:35:25.324 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: nose 2019-12-09T09:35:25.414 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed nose-1.3.7 2019-12-09T09:35:25.970 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:35:26.693 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting setuptools==32.3.1 2019-12-09T09:35:26.739 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl (479kB) 2019-12-09T09:35:26.892 INFO:teuthology.orchestra.run.smithi033.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T09:35:26.892 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: setuptools 2019-12-09T09:35:26.892 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T09:35:26.907 INFO:teuthology.orchestra.run.smithi033.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T09:35:26.916 INFO:teuthology.orchestra.run.smithi033.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T09:35:27.045 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed setuptools-32.3.1 2019-12-09T09:35:27.553 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T09:35:27.942 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting PyYAML 2019-12-09T09:35:27.984 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB) 2019-12-09T09:35:28.597 INFO:teuthology.orchestra.run.smithi033.stdout:Requirement already up-to-date: nose>=1.0.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (1.3.7) 2019-12-09T09:35:28.735 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting boto!=2.46.0,>=2.6.0 2019-12-09T09:35:28.746 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB) 2019-12-09T09:35:29.057 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting bunch>=1.0.0 2019-12-09T09:35:29.068 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/ef/bf/a4cf1779a4ffb4f610903fa08e15d1f4a8a2f4e3353a02afbe097c5bf4a8/bunch-1.0.1.tar.gz 2019-12-09T09:35:29.732 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting gevent>=1.0 2019-12-09T09:35:29.750 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (5.0MB) 2019-12-09T09:35:30.229 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting httplib2 2019-12-09T09:35:30.245 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/ce/2e/87461bfbb7e561203b759b3f7f639e2144226604372830d00a8279960ae1/httplib2-0.14.0.tar.gz (219kB) 2019-12-09T09:35:31.621 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting lxml 2019-12-09T09:35:31.643 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl (5.7MB) 2019-12-09T09:35:32.271 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T09:35:32.282 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T09:35:32.347 INFO:teuthology.orchestra.run.smithi033.stdout:Building wheels for collected packages: PyYAML, bunch, httplib2 2019-12-09T09:35:32.348 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for PyYAML (setup.py): started 2019-12-09T09:35:32.955 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for PyYAML (setup.py): finished with status 'done' 2019-12-09T09:35:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: Created wheel for PyYAML: filename=PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl size=46299 sha256=9fe11fed3b972bc36466e119fa1a2b381c0e76359a4b83f0d029125e791260bf 2019-12-09T09:35:32.956 INFO:teuthology.orchestra.run.smithi033.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67 2019-12-09T09:35:32.971 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for bunch (setup.py): started 2019-12-09T09:35:33.327 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for bunch (setup.py): finished with status 'done' 2019-12-09T09:35:33.327 INFO:teuthology.orchestra.run.smithi033.stdout: Created wheel for bunch: filename=bunch-1.0.1-cp27-none-any.whl size=8347 sha256=9625e9824c0c4996fcbcfa73f777063d0937c12f1e23b1cbbe6db25872695ddd 2019-12-09T09:35:33.327 INFO:teuthology.orchestra.run.smithi033.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c 2019-12-09T09:35:33.332 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for httplib2 (setup.py): started 2019-12-09T09:35:33.659 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for httplib2 (setup.py): finished with status 'done' 2019-12-09T09:35:33.660 INFO:teuthology.orchestra.run.smithi033.stdout: Created wheel for httplib2: filename=httplib2-0.14.0-cp27-none-any.whl size=96742 sha256=8c2d75ebe232b22ea95cc7f8b087f08059b9af52e2c32b2123a565fcc98d0e3e 2019-12-09T09:35:33.660 INFO:teuthology.orchestra.run.smithi033.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488 2019-12-09T09:35:33.670 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully built PyYAML bunch httplib2 2019-12-09T09:35:33.801 INFO:teuthology.orchestra.run.smithi033.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T09:35:33.801 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: PyYAML, boto, bunch, greenlet, gevent, httplib2, lxml 2019-12-09T09:35:33.802 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: PyYAML 3.10 2019-12-09T09:35:33.802 INFO:teuthology.orchestra.run.smithi033.stdout: Not uninstalling pyyaml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T09:35:33.802 INFO:teuthology.orchestra.run.smithi033.stdout: Can't uninstall 'PyYAML'. No files were found to uninstall. 2019-12-09T09:35:34.835 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: lxml 3.2.1 2019-12-09T09:35:34.844 INFO:teuthology.orchestra.run.smithi033.stdout: Not uninstalling lxml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T09:35:34.844 INFO:teuthology.orchestra.run.smithi033.stdout: Can't uninstall 'lxml'. No files were found to uninstall. 2019-12-09T09:35:34.908 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 bunch-1.0.1 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 lxml-4.4.2 2019-12-09T09:35:35.372 INFO:teuthology.orchestra.run.smithi033.stdout:running develop 2019-12-09T09:35:35.460 INFO:teuthology.orchestra.run.smithi033.stdout:running egg_info 2019-12-09T09:35:35.461 INFO:teuthology.orchestra.run.smithi033.stdout:creating ragweed.egg-info 2019-12-09T09:35:35.461 INFO:teuthology.orchestra.run.smithi033.stdout:writing requirements to ragweed.egg-info/requires.txt 2019-12-09T09:35:35.461 INFO:teuthology.orchestra.run.smithi033.stdout:writing ragweed.egg-info/PKG-INFO 2019-12-09T09:35:35.461 INFO:teuthology.orchestra.run.smithi033.stdout:writing top-level names to ragweed.egg-info/top_level.txt 2019-12-09T09:35:35.461 INFO:teuthology.orchestra.run.smithi033.stdout:writing dependency_links to ragweed.egg-info/dependency_links.txt 2019-12-09T09:35:35.462 INFO:teuthology.orchestra.run.smithi033.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:35:35.462 INFO:teuthology.orchestra.run.smithi033.stdout:reading manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:35:35.463 INFO:teuthology.orchestra.run.smithi033.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T09:35:35.465 INFO:teuthology.orchestra.run.smithi033.stdout:running build_ext 2019-12-09T09:35:35.465 INFO:teuthology.orchestra.run.smithi033.stdout:Creating /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/ragweed.egg-link (link to .) 2019-12-09T09:35:35.466 INFO:teuthology.orchestra.run.smithi033.stdout:Adding ragweed 0.0.1 to easy-install.pth file 2019-12-09T09:35:35.466 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.466 INFO:teuthology.orchestra.run.smithi033.stdout:Installed /home/ubuntu/cephtest/ragweed 2019-12-09T09:35:35.468 INFO:teuthology.orchestra.run.smithi033.stdout:Processing dependencies for ragweed==0.0.1 2019-12-09T09:35:35.472 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for isodate>=0.4.4 2019-12-09T09:35:35.472 INFO:teuthology.orchestra.run.smithi033.stdout:Reading https://pypi.python.org/simple/isodate/ 2019-12-09T09:35:35.570 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading https://files.pythonhosted.org/packages/b1/80/fb8c13a4cd38eb5021dc3741a9e588e4d1de88d895c1910c6fc8a08b7a70/isodate-0.6.0.tar.gz#sha256=2e364a3d5759479cdb2d37cce6b9376ea504db2ff90252a2e5b7cc89cc9ff2d8 2019-12-09T09:35:35.619 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: isodate 0.6.0 2019-12-09T09:35:35.620 INFO:teuthology.orchestra.run.smithi033.stdout:Processing isodate-0.6.0.tar.gz 2019-12-09T09:35:35.627 INFO:teuthology.orchestra.run.smithi033.stdout:Writing /tmp/easy_install-XR2JFy/isodate-0.6.0/setup.cfg 2019-12-09T09:35:35.627 INFO:teuthology.orchestra.run.smithi033.stdout:Running isodate-0.6.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-XR2JFy/isodate-0.6.0/egg-dist-tmp-kbawbl 2019-12-09T09:35:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:zip_safe flag not set; analyzing archive contents... 2019-12-09T09:35:35.925 INFO:teuthology.orchestra.run.smithi033.stdout:Moving isodate-0.6.0-py2.7.egg to /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:35:35.926 INFO:teuthology.orchestra.run.smithi033.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T09:35:35.927 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.927 INFO:teuthology.orchestra.run.smithi033.stdout:Installed /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/isodate-0.6.0-py2.7.egg 2019-12-09T09:35:35.951 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for gevent==1.4.0 2019-12-09T09:35:35.951 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: gevent 1.4.0 2019-12-09T09:35:35.951 INFO:teuthology.orchestra.run.smithi033.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T09:35:35.951 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.952 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:35:35.952 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for bunch==1.0.1 2019-12-09T09:35:35.952 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: bunch 1.0.1 2019-12-09T09:35:35.952 INFO:teuthology.orchestra.run.smithi033.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T09:35:35.953 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.953 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:35:35.953 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for PyYAML==5.2 2019-12-09T09:35:35.954 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: PyYAML 5.2 2019-12-09T09:35:35.954 INFO:teuthology.orchestra.run.smithi033.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T09:35:35.955 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.955 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:35:35.955 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for boto==2.49.0 2019-12-09T09:35:35.955 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: boto 2.49.0 2019-12-09T09:35:35.955 INFO:teuthology.orchestra.run.smithi033.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T09:35:35.956 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.956 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:35:35.956 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for six==1.9.0 2019-12-09T09:35:35.957 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: six 1.9.0 2019-12-09T09:35:35.957 INFO:teuthology.orchestra.run.smithi033.stdout:Adding six 1.9.0 to easy-install.pth file 2019-12-09T09:35:35.957 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.957 INFO:teuthology.orchestra.run.smithi033.stdout:Using /usr/lib/python2.7/site-packages 2019-12-09T09:35:35.966 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for greenlet==0.4.15 2019-12-09T09:35:35.967 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: greenlet 0.4.15 2019-12-09T09:35:35.967 INFO:teuthology.orchestra.run.smithi033.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T09:35:35.967 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:35:35.967 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T09:35:35.968 INFO:teuthology.orchestra.run.smithi033.stdout:Finished processing dependencies for ragweed==0.0.1 2019-12-09T09:35:35.989 INFO:teuthology.orchestra.run.smithi033:> cat > /home/ubuntu/cephtest/archive/ragweed.client.0.conf 2019-12-09T09:35:36.053 INFO:tasks.ragweed:Configuring boto... 2019-12-09T09:35:36.054 INFO:teuthology.orchestra.run.smithi033:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T09:35:36.136 INFO:teuthology.orchestra.run.smithi033:ragweed tests against rgw> RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw' 2019-12-09T09:36:05.051 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_multipart_defer_complete.test ... ok 2019-12-09T09:36:29.310 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_multipart_defer_update_complete.test ... ok 2019-12-09T09:36:55.480 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_multipart_index_versioning.test ... ok 2019-12-09T09:37:18.245 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_multipart_simple.test ... ok 2019-12-09T09:37:42.189 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_obj_storage_class.test ... ok 2019-12-09T09:37:45.770 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:09:07.240 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:45.835 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:09:07.305 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:45.840 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:09:07.310 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:45.882 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:09:07.352 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:45.925 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:09:07.416 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:46.010 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:09:07.502 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:46.489 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:09:07.959 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:46.490 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:09:07.960 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:37:46.631 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:09:08.123 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:38:50.427 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_obj_storage_class_copy.test ... ok 2019-12-09T09:39:49.413 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart.test ... ok 2019-12-09T09:40:14.618 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:11:36.090 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.627 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:11:36.099 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.632 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:11:36.105 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.634 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:11:36.128 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.637 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:11:36.110 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.639 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:11:36.133 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.742 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:11:36.214 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.753 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:11:36.226 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:40:14.775 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:11:36.269 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:41:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart_copy.test ... ok 2019-12-09T09:42:18.870 INFO:teuthology.orchestra.run.smithi033.stderr:ragweed.tests.tests.r_test_small_obj_data.test ... ok 2019-12-09T09:42:18.871 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T09:42:18.871 INFO:teuthology.orchestra.run.smithi033.stderr:---------------------------------------------------------------------- 2019-12-09T09:42:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:Ran 9 tests in 402.330s 2019-12-09T09:42:18.872 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T09:42:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:OK 2019-12-09T09:42:18.893 INFO:tasks.ragweed:Cleaning up boto... 2019-12-09T09:42:18.893 INFO:teuthology.orchestra.run.smithi033:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T09:42:18.953 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2019-12-09T09:42:19.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.095+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.130+0000 7f439756fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:19.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.130+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.137+0000 7f439756fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.137+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.145+0000 7f439756fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.145+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.192+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.397+0000 7f439756fec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:42:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.397+0000 7f439756fec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:42:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.397+0000 7f439756fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:42:19.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.397+0000 7f439756fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:42:19.387 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.397+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.414+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:19.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.414+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.419 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.429+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:42:19.419 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.429+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.436 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.446+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:19.436 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.447+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.467+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:42:19.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.467+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.477+0000 7f439756fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:19.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.477+0000 7f439756fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:42:19.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.477+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.486+0000 7f439756fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:19.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.486+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.501+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.501+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.506 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.517+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:42:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.517+0000 7f439756fec0 20 zonegroup default 2019-12-09T09:42:19.507 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.517+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.527+0000 7f439756fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.527+0000 7f439756fec0 10 Cannot find current period zone using local zone 2019-12-09T09:42:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.527+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.526 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.536+0000 7f439756fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:19.526 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.536+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.542 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.553+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:19.542 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.553+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:19.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.572+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:42:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.572+0000 7f439756fec0 20 zone default found 2019-12-09T09:42:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.572+0000 7f439756fec0 20 started sync module instance, tier type = 2019-12-09T09:42:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:19.572+0000 7f439756fec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:42:20.073 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.084+0000 7f439756fec0 20 add_watcher() i=0 2019-12-09T09:42:20.073 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.084+0000 7f439756fec0 20 add_watcher() i=1 2019-12-09T09:42:20.073 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.084+0000 7f439756fec0 20 add_watcher() i=2 2019-12-09T09:42:20.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.084+0000 7f439756fec0 20 add_watcher() i=3 2019-12-09T09:42:20.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.084+0000 7f439756fec0 20 add_watcher() i=4 2019-12-09T09:42:20.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.084+0000 7f439756fec0 20 add_watcher() i=5 2019-12-09T09:42:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.099+0000 7f439756fec0 20 add_watcher() i=6 2019-12-09T09:42:20.117 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.127+0000 7f439756fec0 20 add_watcher() i=7 2019-12-09T09:42:20.117 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.127+0000 7f439756fec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:42:20.117 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.128+0000 7f4354fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:42:20.129 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.140+0000 7f437bfff700 0 -- 172.21.15.33:0/4172955887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558e175f71d0 msgr2=0x558e175f96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:42:20.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:20.177+0000 7f4384799700 0 -- 172.21.15.33:0/457539458 >> [v2:172.21.15.33:6800/9135,v1:172.21.15.33:6801/9135] conn(0x7f42c8022470 msgr2=0x7f42c8024960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:42:21.527 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.538+0000 7f42c6ffd700 20 reqs_thread_entry: start 2019-12-09T09:42:21.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.538+0000 7f42c4ff9700 20 reqs_thread_entry: start 2019-12-09T09:42:21.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.538+0000 7f439756fec0 20 init_complete bucket index max shards: 0 2019-12-09T09:42:21.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.539+0000 7f42baffd700 20 reqs_thread_entry: start 2019-12-09T09:42:21.529 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.539+0000 7f439756fec0 20 get_system_obj_state: rctx=0x7ffd8e5570f8 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x558e17670c00 s->prefetch_data=0 2019-12-09T09:42:21.529 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.539+0000 7f439756fec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T09:42:21.542 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.553+0000 7f439756fec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x16 2019-12-09T09:42:21.542 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.553+0000 7f439756fec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:42:21.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.553+0000 7f439756fec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:21.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.553+0000 7f439756fec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:21.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.553+0000 7f439756fec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T09:42:21.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.553+0000 7f439756fec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.572+0000 7f439756fec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T09:42:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.572+0000 7f439756fec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x11 2019-12-09T09:42:21.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.572+0000 7f439756fec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T09:42:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.579+0000 7f439756fec0 10 removing key index: LZXFPEAOSHYKKAQLAIRH 2019-12-09T09:42:21.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.579+0000 7f439756fec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=170 2019-12-09T09:42:21.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.594+0000 7f4379ffb700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 94068765762224 notifier 4490 bl.length()=170 2019-12-09T09:42:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.742+0000 7f439756fec0 10 removing email index: sysuser.client.0+test@test.test 2019-12-09T09:42:21.732 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.742+0000 7f439756fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=182 2019-12-09T09:42:21.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:21.751+0000 7f4379ffb700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 94068765758848 notifier 4490 bl.length()=182 2019-12-09T09:42:22.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.102+0000 7f439756fec0 10 removing user buckets index 2019-12-09T09:42:22.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.102+0000 7f439756fec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=173 2019-12-09T09:42:22.145 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.156+0000 7f4379ffb700 10 RGWWatcher::handle_notify() notify_id 579820584960 cookie 94068765754288 notifier 4490 bl.length()=173 2019-12-09T09:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.229+0000 7f439756fec0 10 removing user index: sysuser.client.0 2019-12-09T09:42:22.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.229+0000 7f439756fec0 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2019-12-09T09:42:22.219 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.229+0000 7f439756fec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=165 2019-12-09T09:42:22.226 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.237+0000 7f4379ffb700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94068765752592 notifier 4490 bl.length()=165 2019-12-09T09:42:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.968+0000 7f439756fec0 20 remove_watcher() i=0 2019-12-09T09:42:22.957 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:22.968+0000 7f439756fec0 2 removed watcher, disabling cache 2019-12-09T09:42:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.016+0000 7f439756fec0 20 remove_watcher() i=1 2019-12-09T09:42:23.047 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.057+0000 7f439756fec0 20 remove_watcher() i=2 2019-12-09T09:42:23.085 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.096+0000 7f439756fec0 20 remove_watcher() i=3 2019-12-09T09:42:23.133 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.143+0000 7f439756fec0 20 remove_watcher() i=4 2019-12-09T09:42:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.190+0000 7f439756fec0 20 remove_watcher() i=5 2019-12-09T09:42:23.229 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.240+0000 7f439756fec0 20 remove_watcher() i=6 2019-12-09T09:42:23.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.289+0000 7f439756fec0 20 remove_watcher() i=7 2019-12-09T09:42:23.286 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2019-12-09T09:42:23.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.410+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.451+0000 7fe5f9a1aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:23.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.451+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.462+0000 7fe5f9a1aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.462+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.470+0000 7fe5f9a1aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:23.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.470+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.485 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.496+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.703+0000 7fe5f9a1aec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:42:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.703+0000 7fe5f9a1aec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:42:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.703+0000 7fe5f9a1aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:42:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.703+0000 7fe5f9a1aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:42:23.693 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.703+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.722+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:23.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.722+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.721 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.732+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:42:23.722 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.732+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.747+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:23.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.747+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.759+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:42:23.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.759+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.770+0000 7fe5f9a1aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.770+0000 7fe5f9a1aec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:42:23.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.770+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.778+0000 7fe5f9a1aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.778+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.793+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:23.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.793+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.817+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:42:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.817+0000 7fe5f9a1aec0 20 zonegroup default 2019-12-09T09:42:23.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.818+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.812 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.823+0000 7fe5f9a1aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:23.812 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.823+0000 7fe5f9a1aec0 10 Cannot find current period zone using local zone 2019-12-09T09:42:23.812 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.823+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.827 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.832+0000 7fe5f9a1aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:42:23.828 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.832+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.848+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:42:23.838 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.848+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.867+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:42:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.867+0000 7fe5f9a1aec0 20 zone default found 2019-12-09T09:42:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.867+0000 7fe5f9a1aec0 20 started sync module instance, tier type = 2019-12-09T09:42:23.857 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:23.867+0000 7fe5f9a1aec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:42:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.242+0000 7fe5f9a1aec0 20 add_watcher() i=0 2019-12-09T09:42:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.242+0000 7fe5f9a1aec0 20 add_watcher() i=1 2019-12-09T09:42:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.242+0000 7fe5f9a1aec0 20 add_watcher() i=2 2019-12-09T09:42:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.242+0000 7fe5f9a1aec0 20 add_watcher() i=3 2019-12-09T09:42:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.242+0000 7fe5f9a1aec0 20 add_watcher() i=4 2019-12-09T09:42:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.242+0000 7fe5f9a1aec0 20 add_watcher() i=5 2019-12-09T09:42:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.242+0000 7fe5f9a1aec0 20 add_watcher() i=6 2019-12-09T09:42:24.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.277+0000 7fe5f9a1aec0 20 add_watcher() i=7 2019-12-09T09:42:24.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.277+0000 7fe5f9a1aec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:42:24.267 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.277+0000 7fe5b77de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:42:24.267 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.278+0000 7fe5e7445700 0 -- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x56428a696d20 msgr2=0x56428a699ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T09:42:24.310 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:24.321+0000 7fe5e7445700 0 -- 172.21.15.33:0/516212828 >> [v2:172.21.15.33:6800/9135,v1:172.21.15.33:6801/9135] conn(0x7fe528022480 msgr2=0x7fe528024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:42:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.503+0000 7fe525ffb700 20 reqs_thread_entry: start 2019-12-09T09:42:25.492 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.503+0000 7fe51f7fe700 20 reqs_thread_entry: start 2019-12-09T09:42:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.503+0000 7fe5f9a1aec0 20 init_complete bucket index max shards: 0 2019-12-09T09:42:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.504+0000 7fe51dffb700 20 reqs_thread_entry: start 2019-12-09T09:42:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.504+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffeb8 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x56428a7199d0 s->prefetch_data=0 2019-12-09T09:42:25.494 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.504+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T09:42:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.526+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x16 2019-12-09T09:42:25.515 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.526+0000 7fe5f9a1aec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:42:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.526+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.526+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.526+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T09:42:25.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.526+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:25.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.547+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T09:42:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.547+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x11 2019-12-09T09:42:25.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.547+0000 7fe5f9a1aec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T09:42:25.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.569+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-conf state=0x56428a71b2f0 s->prefetch_data=0 2019-12-09T09:42:25.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.569+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2019-12-09T09:42:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.583+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x6 2019-12-09T09:42:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.583+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T09:42:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.583+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.583+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:25.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.583+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:25.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.583+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:25.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.593+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=200 2019-12-09T09:42:25.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.593+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x1 2019-12-09T09:42:25.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.593+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T09:42:25.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.593+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 state=0x56428a71fb30 s->prefetch_data=0 2019-12-09T09:42:25.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.593+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 : miss 2019-12-09T09:42:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 info.flags=0x16 2019-12-09T09:42:25.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 to cache LRU end 2019-12-09T09:42:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:25.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:25.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.615+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.640+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=292 2019-12-09T09:42:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.640+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 info.flags=0x13 2019-12-09T09:42:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.640+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 to cache LRU end 2019-12-09T09:42:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.640+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.640+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.640+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:25.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.655+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-conf[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1]) start [] num_entries 1100 2019-12-09T09:42:25.672 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete[] 2019-12-09T09:42:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete[] 2019-12-09T09:42:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning[] 2019-12-09T09:42:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_simple[] 2019-12-09T09:42:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class[] 2019-12-09T09:42:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy[] 2019-12-09T09:42:25.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart[] 2019-12-09T09:42:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy[] 2019-12-09T09:42:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_small_obj_data[] 2019-12-09T09:42:25.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.683+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.695+0000 7fe5f9a1aec0 10 manifest: total_size = 3909 2019-12-09T09:42:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.695+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.13 2019-12-09T09:42:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.695+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.695+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.695+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.925+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.935+0000 7fe5f9a1aec0 10 manifest: total_size = 3887 2019-12-09T09:42:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.935+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.30 2019-12-09T09:42:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.935+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.935+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:25.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:25.935+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:26.187 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.197+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.217+0000 7fe5f9a1aec0 10 manifest: total_size = 9698 2019-12-09T09:42:26.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.217+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.56 2019-12-09T09:42:26.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.217+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.217+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:26.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.217+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:26.499 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.510+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.521+0000 7fe5f9a1aec0 10 manifest: total_size = 5731 2019-12-09T09:42:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.521+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.72 2019-12-09T09:42:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.521+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.521+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:26.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.521+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:26.729 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.740+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.792+0000 7fe5f9a1aec0 10 manifest: total_size = 10467 2019-12-09T09:42:26.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.792+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.87 2019-12-09T09:42:26.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.792+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.792+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:26.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.792+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:26.929 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.940+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.949+0000 7fe5f9a1aec0 10 manifest: total_size = 6047 2019-12-09T09:42:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.949+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.120 2019-12-09T09:42:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.949+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.949+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:26.949+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:27.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.095+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.118+0000 7fe5f9a1aec0 10 manifest: total_size = 6188 2019-12-09T09:42:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.118+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.189 2019-12-09T09:42:27.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.118+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.118+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:27.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.118+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:27.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.244+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:27.254 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.264+0000 7fe5f9a1aec0 10 manifest: total_size = 6127 2019-12-09T09:42:27.254 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.264+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.246 2019-12-09T09:42:27.254 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.264+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:27.254 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.264+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:27.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.264+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.458+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.474+0000 7fe5f9a1aec0 10 manifest: total_size = 6761 2019-12-09T09:42:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.474+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.322 2019-12-09T09:42:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.474+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x56428a722ae8 s->prefetch_data=0 2019-12-09T09:42:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.474+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ae8 2019-12-09T09:42:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.474+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 2019-12-09T09:42:27.641 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.652+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-conf[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1]) start _multipart_[] num_entries 1001 2019-12-09T09:42:27.641 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.652+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.715+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:27.786 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.797+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-conf state=0x56428a724050 s->prefetch_data=0 2019-12-09T09:42:27.787 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.797+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.807+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x16 2019-12-09T09:42:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.807+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T09:42:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.807+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.807+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.807+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:27.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.807+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-conf from cache 2019-12-09T09:42:27.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.807+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=162 2019-12-09T09:42:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:27.845+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584960 cookie 94843789927760 notifier 4502 bl.length()=162 2019-12-09T09:42:28.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:28.275+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.1 from cache 2019-12-09T09:42:28.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:28.275+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=219 2019-12-09T09:42:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:28.294+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94843789933584 notifier 4502 bl.length()=219 2019-12-09T09:42:29.101 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.112+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x56428a71b380 s->prefetch_data=0 2019-12-09T09:42:29.101 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.112+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2019-12-09T09:42:29.111 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.122+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x6 2019-12-09T09:42:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.122+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T09:42:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.122+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.122+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.122+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:29.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.122+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.134+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=229 2019-12-09T09:42:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.134+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x1 2019-12-09T09:42:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.134+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T09:42:29.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.134+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 state=0x56428a723450 s->prefetch_data=0 2019-12-09T09:42:29.124 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.134+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 : miss 2019-12-09T09:42:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 info.flags=0x16 2019-12-09T09:42:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 to cache LRU end 2019-12-09T09:42:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:29.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:29.136 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:29.136 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:29.136 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:29.136 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.145+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.161+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=321 2019-12-09T09:42:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.161+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 info.flags=0x13 2019-12-09T09:42:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.161+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 to cache LRU end 2019-12-09T09:42:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.161+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.161+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.161+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 2019-12-09T09:42:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.176+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-complete.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2]) start [] num_entries 1100 2019-12-09T09:42:29.182 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.190+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T09:42:29.183 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.190+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x56428a723718 s->prefetch_data=0 2019-12-09T09:42:29.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.199+0000 7fe5f9a1aec0 10 manifest: total_size = 1848 2019-12-09T09:42:29.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.199+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.20 2019-12-09T09:42:29.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.199+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x56428a723718 s->prefetch_data=0 2019-12-09T09:42:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.199+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a723718 2019-12-09T09:42:29.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.199+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 2019-12-09T09:42:29.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:42:29.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-12-09T09:42:29.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T09:42:29.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:42:29.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-12-09T09:42:29.342 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-12-09T09:42:29.342 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:42:29.342 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=515 2019-12-09T09:42:29.342 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-12-09T09:42:29.342 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-12-09T09:42:29.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4195849 part_ofs=1545 rule->part_size=303 2019-12-09T09:42:29.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-12-09T09:42:29.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.351+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.20 2019-12-09T09:42:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.476+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-complete.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2]) start _multipart_[] num_entries 1001 2019-12-09T09:42:29.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.476+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:29.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.507+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:29.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.578+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x56428a723450 s->prefetch_data=0 2019-12-09T09:42:29.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.578+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.588+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x16 2019-12-09T09:42:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.588+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T09:42:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.588+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.588+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.588+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.588+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2019-12-09T09:42:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.588+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=191 2019-12-09T09:42:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:29.657+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584960 cookie 94843789929744 notifier 4502 bl.length()=191 2019-12-09T09:42:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.160+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2 from cache 2019-12-09T09:42:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.160+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=248 2019-12-09T09:42:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.180+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 94843789940208 notifier 4502 bl.length()=248 2019-12-09T09:42:30.693 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.704+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x56428a71b380 s->prefetch_data=0 2019-12-09T09:42:30.694 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.704+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2019-12-09T09:42:30.710 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.721+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x6 2019-12-09T09:42:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.721+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T09:42:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.721+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.721+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.721+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:30.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.721+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.739+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=236 2019-12-09T09:42:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.739+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x1 2019-12-09T09:42:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.739+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T09:42:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.739+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:30.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.739+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 : miss 2019-12-09T09:42:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 info.flags=0x16 2019-12-09T09:42:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 to cache LRU end 2019-12-09T09:42:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.761+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:30.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.791+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=328 2019-12-09T09:42:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.791+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 info.flags=0x13 2019-12-09T09:42:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.791+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 to cache LRU end 2019-12-09T09:42:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.791+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:30.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.791+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:30.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.791+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 2019-12-09T09:42:30.805 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.808+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-update-complete.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3]) start [] num_entries 1100 2019-12-09T09:42:30.817 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.828+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T09:42:30.817 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.828+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.844+0000 7fe5f9a1aec0 10 manifest: total_size = 886297 2019-12-09T09:42:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.844+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.40 2019-12-09T09:42:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.844+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:30.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.844+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:30.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.844+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 2019-12-09T09:42:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:42:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-09T09:42:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-09T09:42:30.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:42:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-09T09:42:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2019-12-09T09:42:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T09:42:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2019-12-09T09:42:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2019-12-09T09:42:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2019-12-09T09:42:30.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2019-12-09T09:42:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2019-12-09T09:42:30.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:30.968+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.40 2019-12-09T09:42:31.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.102+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-update-complete.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3]) start _multipart_[] num_entries 1001 2019-12-09T09:42:31.092 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.102+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:31.110 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.120+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.186+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x56428a722320 s->prefetch_data=0 2019-12-09T09:42:31.176 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.186+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.206+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x16 2019-12-09T09:42:31.195 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.206+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T09:42:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.206+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.206+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.206+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.206+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2019-12-09T09:42:31.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.206+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=198 2019-12-09T09:42:31.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.220+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584967 cookie 94843789933584 notifier 4502 bl.length()=198 2019-12-09T09:42:31.657 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.668+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3 from cache 2019-12-09T09:42:31.658 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.668+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=255 2019-12-09T09:42:31.672 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:31.683+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584968 cookie 94843789934672 notifier 4502 bl.length()=255 2019-12-09T09:42:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.345+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.345+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2019-12-09T09:42:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.361+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x6 2019-12-09T09:42:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.361+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T09:42:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.361+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.361+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.361+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.361+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.370+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=231 2019-12-09T09:42:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.370+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x1 2019-12-09T09:42:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.370+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T09:42:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.370+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 state=0x56428a71b380 s->prefetch_data=0 2019-12-09T09:42:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.371+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 : miss 2019-12-09T09:42:32.382 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 info.flags=0x16 2019-12-09T09:42:32.382 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 to cache LRU end 2019-12-09T09:42:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.393+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.413+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=323 2019-12-09T09:42:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.413+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 info.flags=0x13 2019-12-09T09:42:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.413+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 to cache LRU end 2019-12-09T09:42:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.413+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.413+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:32.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.413+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:32.427 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.438+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-index-versioning.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4]) start [] num_entries 1100 2019-12-09T09:42:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.460+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo[lHV0eS1iUhaz1.MNV3CHroKGcSpdU.D] 2019-12-09T09:42:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.460+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:32.470 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.481+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.498+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to cp4mmkczz9hatcede8d2gr3jxoeqz654 2019-12-09T09:42:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.498+0000 7fe5f9a1aec0 20 get_obj_state_impl: setting s->olh_tag to 6h1gdv8w2wfeizdkm8oqb42cagam63hd 2019-12-09T09:42:32.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.498+0000 7fe5f9a1aec0 20 bucket_index_guard_olh_op(): olh_state.olh_tag=6h1gdv8w2wfeizdkm8oqb42cagam63hd 2019-12-09T09:42:32.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.569+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:32.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.629+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.629+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:32.632 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.643+0000 7fe5f9a1aec0 20 olh_log_entry: epoch=3 op=2 key=[] 2019-12-09T09:42:32.632 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.643+0000 7fe5f9a1aec0 20 olh_log_entry: epoch=3 op=3 key=foo[lHV0eS1iUhaz1.MNV3CHroKGcSpdU.D] 2019-12-09T09:42:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.643+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-index-versioning.1:_:lHV0eS1iUhaz1.MNV3CHroKGcSpdU.D_foo state=0x56428a723718 s->prefetch_data=0 2019-12-09T09:42:32.642 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.653+0000 7fe5f9a1aec0 10 manifest: total_size = 26081 2019-12-09T09:42:32.642 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.653+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.55 2019-12-09T09:42:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.653+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-index-versioning.1:_:lHV0eS1iUhaz1.MNV3CHroKGcSpdU.D_foo state=0x56428a723718 s->prefetch_data=0 2019-12-09T09:42:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.653+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a723718 2019-12-09T09:42:32.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.653+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:32.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=4 2019-12-09T09:42:32.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7671 2019-12-09T09:42:32.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=7671 stripe_ofs=7671 part_ofs=7671 rule->part_size=7670 2019-12-09T09:42:32.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=7670 rules.size()=4 2019-12-09T09:42:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4201975 part_ofs=7671 rule->part_size=7670 2019-12-09T09:42:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=15341 stripe_ofs=15341 part_ofs=15341 rule->part_size=7673 2019-12-09T09:42:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=7673 rules.size()=4 2019-12-09T09:42:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4209645 part_ofs=15341 rule->part_size=7673 2019-12-09T09:42:32.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=23014 stripe_ofs=23014 part_ofs=23014 rule->part_size=3067 2019-12-09T09:42:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=3067 rules.size()=4 2019-12-09T09:42:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4217318 part_ofs=23014 rule->part_size=3067 2019-12-09T09:42:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=26081 stripe_ofs=26081 part_ofs=26081 rule->part_size=3067 2019-12-09T09:42:32.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.757+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.55 2019-12-09T09:42:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.952+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:32.952+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:33.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.075+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.113+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 2019-12-09T09:42:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.113+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-index-versioning.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4]) start _multipart_[] num_entries 1001 2019-12-09T09:42:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.113+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.123+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:33.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.190+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x56428a7286a0 s->prefetch_data=0 2019-12-09T09:42:33.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.190+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.207+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x16 2019-12-09T09:42:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.207+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T09:42:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.207+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.207+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:33.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.207+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:33.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.207+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2019-12-09T09:42:33.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.207+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=193 2019-12-09T09:42:33.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.218+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584969 cookie 94843789933584 notifier 4502 bl.length()=193 2019-12-09T09:42:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.813+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4 from cache 2019-12-09T09:42:33.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.813+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=250 2019-12-09T09:42:33.812 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:33.823+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94843789943856 notifier 4502 bl.length()=250 2019-12-09T09:42:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.354+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:34.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.354+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2019-12-09T09:42:34.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.371+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x6 2019-12-09T09:42:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.371+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T09:42:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.371+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.371+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.371+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:34.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.371+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.388+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:42:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.388+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x1 2019-12-09T09:42:34.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.388+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T09:42:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.388+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 state=0x56428a71b380 s->prefetch_data=0 2019-12-09T09:42:34.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.388+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 : miss 2019-12-09T09:42:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 info.flags=0x16 2019-12-09T09:42:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 to cache LRU end 2019-12-09T09:42:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.403+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:34.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.421+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:42:34.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.421+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 info.flags=0x13 2019-12-09T09:42:34.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.421+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 to cache LRU end 2019-12-09T09:42:34.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.421+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:34.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.421+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:34.413 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.421+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 2019-12-09T09:42:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.442+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-simple.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5]) start [] num_entries 1100 2019-12-09T09:42:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.462+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T09:42:34.453 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.463+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:34.469 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.480+0000 7fe5f9a1aec0 10 manifest: total_size = 74198 2019-12-09T09:42:34.469 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.480+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.71 2019-12-09T09:42:34.469 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.480+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:34.470 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.480+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:34.470 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.480+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 2019-12-09T09:42:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:42:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-12-09T09:42:34.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-09T09:42:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:42:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-12-09T09:42:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-12-09T09:42:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T09:42:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=20613 2019-12-09T09:42:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-12-09T09:42:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-12-09T09:42:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4256143 part_ofs=61839 rule->part_size=12359 2019-12-09T09:42:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-12-09T09:42:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.594+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.71 2019-12-09T09:42:34.718 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.729+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-simple.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5]) start _multipart_[] num_entries 1001 2019-12-09T09:42:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.729+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:34.738 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.749+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:34.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.820+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x56428a7286a0 s->prefetch_data=0 2019-12-09T09:42:34.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.820+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.832+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x16 2019-12-09T09:42:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.832+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T09:42:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.832+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.832+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.832+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.832+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2019-12-09T09:42:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.832+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=183 2019-12-09T09:42:34.842 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:34.852+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584970 cookie 94843789934672 notifier 4502 bl.length()=183 2019-12-09T09:42:35.757 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:35.768+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5 from cache 2019-12-09T09:42:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:35.768+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=240 2019-12-09T09:42:35.778 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:35.781+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 94843789927760 notifier 4502 bl.length()=240 2019-12-09T09:42:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.349+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:36.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.349+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2019-12-09T09:42:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.368+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x6 2019-12-09T09:42:36.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.368+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T09:42:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.368+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.368+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.368+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:36.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.368+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:36.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.383+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=227 2019-12-09T09:42:36.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.383+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x1 2019-12-09T09:42:36.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.383+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T09:42:36.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.383+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 state=0x56428a728c70 s->prefetch_data=0 2019-12-09T09:42:36.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.383+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 : miss 2019-12-09T09:42:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 info.flags=0x16 2019-12-09T09:42:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 to cache LRU end 2019-12-09T09:42:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:36.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:36.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:36.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:36.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.394+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.415+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=319 2019-12-09T09:42:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.415+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 info.flags=0x13 2019-12-09T09:42:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.415+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 to cache LRU end 2019-12-09T09:42:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.415+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:36.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.415+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:36.405 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.415+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:36.416 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.428+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-copy.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7]) start [] num_entries 1100 2019-12-09T09:42:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:42:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T09:42:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T09:42:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T09:42:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:42:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T09:42:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T09:42:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T09:42:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:42:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T09:42:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T09:42:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T09:42:36.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.450+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:36.458 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.469+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.469+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.108 2019-12-09T09:42:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.469+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.469+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:36.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.469+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:36.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.622+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:36.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.622+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:36.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.622+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:36.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.622+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:36.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.622+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.108 2019-12-09T09:42:36.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.709+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.726+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.726+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.109 2019-12-09T09:42:36.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.726+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.726+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:36.716 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.726+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.866+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.866+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.866+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.866+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:36.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.866+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.109 2019-12-09T09:42:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.959+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.972+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.972+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _UNq2FOcaTDLxIiZZGjFHLi9SnuC1jWP 2019-12-09T09:42:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.972+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.972+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:36.972+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.105+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.105+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.105+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.105+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.105+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: _UNq2FOcaTDLxIiZZGjFHLi9SnuC1jWP 2019-12-09T09:42:37.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.213+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.226+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.226+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _A5km80k2Kt0ho-8q_rLwMu7kftpb5xi 2019-12-09T09:42:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.226+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:37.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.226+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:37.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.226+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:37.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.355+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:37.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.355+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:37.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.355+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: _A5km80k2Kt0ho-8q_rLwMu7kftpb5xi 2019-12-09T09:42:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.563+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.572+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.572+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.112 2019-12-09T09:42:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.572+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.572+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.572+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:37.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.759+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:37.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.759+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:37.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.759+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:37.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.759+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:37.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.759+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.112 2019-12-09T09:42:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.837+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.847+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.847+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _hyK_gu_ODRFjlIXdRixGOxDzNHgdjXm 2019-12-09T09:42:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.847+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.847+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.847+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:37.948 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.959+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:37.949 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.959+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:37.949 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.959+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:37.949 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.959+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:37.949 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:37.959+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _hyK_gu_ODRFjlIXdRixGOxDzNHgdjXm 2019-12-09T09:42:38.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.014+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.036+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:38.027 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.036+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.114 2019-12-09T09:42:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.036+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.036+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.036+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.220+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:38.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.220+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.220+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.220+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.220+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.114 2019-12-09T09:42:38.410 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.421+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:38.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.443+0000 7fe5f9a1aec0 10 manifest: total_size = 5242880 2019-12-09T09:42:38.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.443+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _QvBbFiVuaUosXs1TuQ9-Ha1MCG_iJ34 2019-12-09T09:42:38.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.443+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.443+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:38.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.443+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.565+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.565+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T09:42:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.565+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: _QvBbFiVuaUosXs1TuQ9-Ha1MCG_iJ34 2019-12-09T09:42:38.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.638+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:38.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.649+0000 7fe5f9a1aec0 10 manifest: total_size = 2043 2019-12-09T09:42:38.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.649+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.116 2019-12-09T09:42:38.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.649+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:38.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.649+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:38.639 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.649+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:38.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.747+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.770+0000 7fe5f9a1aec0 10 manifest: total_size = 2043 2019-12-09T09:42:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.770+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _5YyNuvRzLNYPh9d7rbiUS7fa8vpJYBr 2019-12-09T09:42:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.770+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.770+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.770+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:38.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.933+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.933+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=2043 stripe_ofs=2043 part_ofs=0 rule->part_size=0 2019-12-09T09:42:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:38.933+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _5YyNuvRzLNYPh9d7rbiUS7fa8vpJYBr 2019-12-09T09:42:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.005+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.019+0000 7fe5f9a1aec0 10 manifest: total_size = 2043 2019-12-09T09:42:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.019+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _5kkz9bPGQBSaFgRoxedGxT1h9_JDSa4 2019-12-09T09:42:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.019+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:39.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.019+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.019+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.118+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.118+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=2043 stripe_ofs=2043 part_ofs=0 rule->part_size=0 2019-12-09T09:42:39.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.118+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: _5kkz9bPGQBSaFgRoxedGxT1h9_JDSa4 2019-12-09T09:42:39.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.167+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.176+0000 7fe5f9a1aec0 10 manifest: total_size = 2043 2019-12-09T09:42:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.176+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _VmWVGxBEXwh9k8Ke8tEJe4ttgdsV5KL 2019-12-09T09:42:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.176+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.176+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:39.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.176+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 2019-12-09T09:42:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.265+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-copy.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7]) start _multipart_[] num_entries 1001 2019-12-09T09:42:39.254 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.265+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:39.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.344+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.422+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x56428a727650 s->prefetch_data=0 2019-12-09T09:42:39.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.422+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.432+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x16 2019-12-09T09:42:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.432+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T09:42:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.432+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.432+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.432+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.432+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2019-12-09T09:42:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.432+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=189 2019-12-09T09:42:39.435 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.446+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584971 cookie 94843789933584 notifier 4502 bl.length()=189 2019-12-09T09:42:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.783+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7 from cache 2019-12-09T09:42:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.783+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=246 2019-12-09T09:42:39.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:39.837+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94843789943856 notifier 4502 bl.length()=246 2019-12-09T09:42:40.346 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.357+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:40.346 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.357+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2019-12-09T09:42:40.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.382+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x6 2019-12-09T09:42:40.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.382+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T09:42:40.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.382+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:40.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.382+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:40.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.382+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:40.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.382+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:40.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.395+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=237 2019-12-09T09:42:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.395+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x1 2019-12-09T09:42:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.395+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T09:42:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.395+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 state=0x56428a728a60 s->prefetch_data=0 2019-12-09T09:42:40.387 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.395+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 : miss 2019-12-09T09:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 info.flags=0x16 2019-12-09T09:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 to cache LRU end 2019-12-09T09:42:40.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:40.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.406+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.428+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=329 2019-12-09T09:42:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.428+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 info.flags=0x13 2019-12-09T09:42:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.428+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 to cache LRU end 2019-12-09T09:42:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.428+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.428+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.428+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.443+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9]) start [] num_entries 1100 2019-12-09T09:42:40.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:42:40.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T09:42:40.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T09:42:40.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T09:42:40.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:42:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T09:42:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T09:42:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T09:42:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:42:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T09:42:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T09:42:40.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T09:42:40.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.465+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.476+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.476+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.224 2019-12-09T09:42:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.476+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.476+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.476+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:40.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:40.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:40.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:40.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:40.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:42:40.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:40.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:42:40.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.600+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.224 2019-12-09T09:42:40.651 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.663+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.672+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.672+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.225 2019-12-09T09:42:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.672+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.672+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:40.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.672+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:40.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.775 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:42:40.776 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:42:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.785+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.225 2019-12-09T09:42:40.841 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.852+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:40.850 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.862+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.862+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _8AaBRyOcBy5-yvRhi4b80LcnCFPSdg3 2019-12-09T09:42:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.862+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.862+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:40.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.862+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:40.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.970+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:40.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.970+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:40.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.970+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:40.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.970+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:42:40.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.970+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:40.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.970+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:42:40.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:40.970+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: _8AaBRyOcBy5-yvRhi4b80LcnCFPSdg3 2019-12-09T09:42:41.094 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.105+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.119+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.119+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _ydrmbnPmPMLK1dglQ5asiIJd28-eoUT 2019-12-09T09:42:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.119+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:41.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.119+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:41.109 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.119+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:42:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:41.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.351+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:42:41.342 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.352+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: _ydrmbnPmPMLK1dglQ5asiIJd28-eoUT 2019-12-09T09:42:41.519 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.530+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:41.534 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.544+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:41.534 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.544+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.233 2019-12-09T09:42:41.534 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.544+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x56428a7239e8 s->prefetch_data=0 2019-12-09T09:42:41.534 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.544+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a7239e8 2019-12-09T09:42:41.534 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.545+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:41.635 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:41.635 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:41.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:41.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:41.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:41.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:41.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:41.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:41.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:41.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:41.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:41.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:41.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:42:41.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:41.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:42:41.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.646+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.233 2019-12-09T09:42:41.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.724+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:41.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.739+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:41.729 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.739+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _ruB1BfhKILjAQn_d_fPjx3ws4koTHKa 2019-12-09T09:42:41.729 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.739+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:41.729 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.739+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:41.729 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.739+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.843+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:41.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.843+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:41.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.843+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:41.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.843+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:42:41.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.843+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:41.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.843+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:42:41.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.843+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: _ruB1BfhKILjAQn_d_fPjx3ws4koTHKa 2019-12-09T09:42:41.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.929+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:41.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.949+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:41.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.949+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.235 2019-12-09T09:42:41.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.949+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:41.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.949+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:41.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:41.949+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:42.106 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:42.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:42.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:42:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:42:42.109 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.116+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.235 2019-12-09T09:42:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.216+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.227+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.227+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _Po1ZWhqBuImCoqqbzw0hj3mC8fx4rIx 2019-12-09T09:42:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.227+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.227+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:42.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.227+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:42.430 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.441+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:42.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.441+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:42:42.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.441+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:42.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.441+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T09:42:42.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.441+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: _Po1ZWhqBuImCoqqbzw0hj3mC8fx4rIx 2019-12-09T09:42:42.535 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.546+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.561+0000 7fe5f9a1aec0 10 manifest: total_size = 1140 2019-12-09T09:42:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.561+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.242 2019-12-09T09:42:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.561+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.561+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.561+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:42:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-12-09T09:42:42.649 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T09:42:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:42:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-12-09T09:42:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=110 2019-12-09T09:42:42.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=110 rules.size()=2 2019-12-09T09:42:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=110 2019-12-09T09:42:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=1140 rule->part_size=110 2019-12-09T09:42:42.651 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.659+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.18tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.242 2019-12-09T09:42:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.731+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.730 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.740+0000 7fe5f9a1aec0 10 manifest: total_size = 1140 2019-12-09T09:42:42.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.740+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _rNNl0DHW1__qZTUjFwE5dBaPhDJpZcv 2019-12-09T09:42:42.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.740+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.740+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:42.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.741+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:42.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.854+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:42.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.854+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=0 rule->part_size=0 2019-12-09T09:42:42.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.854+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: _rNNl0DHW1__qZTUjFwE5dBaPhDJpZcv 2019-12-09T09:42:42.911 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.922+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.936+0000 7fe5f9a1aec0 10 manifest: total_size = 1140 2019-12-09T09:42:42.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.936+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to _JICTi0rqsIxW3iMTh9vkigKt4VZhaHJ 2019-12-09T09:42:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.936+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.936+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:42.926 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:42.936+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.048+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.048+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=0 rule->part_size=0 2019-12-09T09:42:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.048+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _JICTi0rqsIxW3iMTh9vkigKt4VZhaHJ 2019-12-09T09:42:43.095 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.106+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.115+0000 7fe5f9a1aec0 10 manifest: total_size = 1140 2019-12-09T09:42:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.115+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.245 2019-12-09T09:42:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.115+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.115+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:43.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.115+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 2019-12-09T09:42:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:42:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-12-09T09:42:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T09:42:43.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T09:42:43.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-12-09T09:42:43.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=110 2019-12-09T09:42:43.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=110 rules.size()=2 2019-12-09T09:42:43.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=110 2019-12-09T09:42:43.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=1140 rule->part_size=110 2019-12-09T09:42:43.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.203+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.245 2019-12-09T09:42:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.279+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9]) start _multipart_[] num_entries 1001 2019-12-09T09:42:43.268 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.279+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:43.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.290+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.360+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.360+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:43.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.378+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x16 2019-12-09T09:42:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.378+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T09:42:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.378+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.378+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.378+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.378+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2019-12-09T09:42:43.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.378+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=199 2019-12-09T09:42:43.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.389+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 94843789936928 notifier 4502 bl.length()=199 2019-12-09T09:42:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.552+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9 from cache 2019-12-09T09:42:43.541 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.552+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=256 2019-12-09T09:42:43.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:43.561+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 94843789944736 notifier 4502 bl.length()=256 2019-12-09T09:42:44.154 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.166+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:44.155 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.166+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2019-12-09T09:42:44.171 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.182+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x6 2019-12-09T09:42:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.182+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T09:42:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.182+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.182+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.182+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:44.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.182+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.199+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=232 2019-12-09T09:42:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.199+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x1 2019-12-09T09:42:44.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.199+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T09:42:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.199+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 state=0x56428a723e30 s->prefetch_data=0 2019-12-09T09:42:44.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.199+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 : miss 2019-12-09T09:42:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 info.flags=0x16 2019-12-09T09:42:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 to cache LRU end 2019-12-09T09:42:44.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:44.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:44.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.210+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.228+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=324 2019-12-09T09:42:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.228+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 info.flags=0x13 2019-12-09T09:42:44.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.228+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 to cache LRU end 2019-12-09T09:42:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.228+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.228+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:44.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.228+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 2019-12-09T09:42:44.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.243+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8]) start [] num_entries 1100 2019-12-09T09:42:44.245 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.256+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:42:44.245 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.256+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:42:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.256+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:42:44.246 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.256+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.286+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.286+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.196 2019-12-09T09:42:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.287+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.287+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.287+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 2019-12-09T09:42:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:42:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:44.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:42:44.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.381+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.196 2019-12-09T09:42:44.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.465+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:44.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.489+0000 7fe5f9a1aec0 10 manifest: total_size = 11534336 2019-12-09T09:42:44.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.489+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.203 2019-12-09T09:42:44.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.489+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:44.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.489+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:44.479 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.489+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 2019-12-09T09:42:44.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:44.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:44.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T09:42:44.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T09:42:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T09:42:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T09:42:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T09:42:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T09:42:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.589+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.25tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.203 2019-12-09T09:42:44.634 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.645+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:44.641 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.653+0000 7fe5f9a1aec0 10 manifest: total_size = 16381 2019-12-09T09:42:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.653+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.210 2019-12-09T09:42:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.653+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.653+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:44.642 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.653+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 2019-12-09T09:42:44.738 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=7674 rules.size()=3 2019-12-09T09:42:44.738 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7674 2019-12-09T09:42:44.738 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=7674 stripe_ofs=7674 part_ofs=7674 rule->part_size=7672 2019-12-09T09:42:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=3 2019-12-09T09:42:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4201978 part_ofs=7674 rule->part_size=7672 2019-12-09T09:42:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=15346 stripe_ofs=15346 part_ofs=15346 rule->part_size=1035 2019-12-09T09:42:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=3 2019-12-09T09:42:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): stripe_ofs=4209650 part_ofs=15346 rule->part_size=1035 2019-12-09T09:42:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=16381 stripe_ofs=16381 part_ofs=16381 rule->part_size=1035 2019-12-09T09:42:44.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.749+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.210 2019-12-09T09:42:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.804+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8]) start _multipart_[] num_entries 1001 2019-12-09T09:42:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.804+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.816+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:44.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.875+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x56428a722a00 s->prefetch_data=0 2019-12-09T09:42:44.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.875+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.885+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x16 2019-12-09T09:42:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.886+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T09:42:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.886+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.886+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:44.876 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.886+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.886+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2019-12-09T09:42:44.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.886+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=194 2019-12-09T09:42:44.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:44.898+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 94843789940208 notifier 4502 bl.length()=194 2019-12-09T09:42:45.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:45.640+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8 from cache 2019-12-09T09:42:45.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:45.640+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=251 2019-12-09T09:42:45.639 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:45.650+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584963 cookie 94843789944736 notifier 4502 bl.length()=251 2019-12-09T09:42:46.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.277+0000 7fe5b77de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:42:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.356+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x56428a723e30 s->prefetch_data=0 2019-12-09T09:42:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.356+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2019-12-09T09:42:46.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.368+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x6 2019-12-09T09:42:46.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.368+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T09:42:46.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.368+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:46.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.368+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:46.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.368+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:46.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.368+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:46.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.381+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=222 2019-12-09T09:42:46.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.381+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x1 2019-12-09T09:42:46.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.381+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T09:42:46.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.381+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 state=0x56428a7236c0 s->prefetch_data=0 2019-12-09T09:42:46.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.381+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 : miss 2019-12-09T09:42:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 info.flags=0x16 2019-12-09T09:42:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 to cache LRU end 2019-12-09T09:42:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.406+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:46.436 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.439+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=314 2019-12-09T09:42:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.439+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 info.flags=0x13 2019-12-09T09:42:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.439+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 to cache LRU end 2019-12-09T09:42:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.439+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.439+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.439+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 2019-12-09T09:42:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.453+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6]) start [] num_entries 1100 2019-12-09T09:42:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.468+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T09:42:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.468+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T09:42:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.468+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T09:42:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.468+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.484+0000 7fe5f9a1aec0 10 manifest: total_size = 10485760 2019-12-09T09:42:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.484+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.84 2019-12-09T09:42:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.484+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.484+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:46.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.484+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 2019-12-09T09:42:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.635+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.635+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.635+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.635+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:42:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.635+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.635+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T09:42:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.635+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.20tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.84 2019-12-09T09:42:46.689 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.696+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.706+0000 7fe5f9a1aec0 10 manifest: total_size = 10485760 2019-12-09T09:42:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.706+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.85 2019-12-09T09:42:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.706+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.706+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.706+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 2019-12-09T09:42:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.825+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.825+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T09:42:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.825+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:46.814 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.825+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T09:42:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.825+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T09:42:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.825+0000 7fe5f9a1aec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T09:42:46.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.825+0000 7fe5f9a1aec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.85 2019-12-09T09:42:46.907 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.919+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:46.921 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.933+0000 7fe5f9a1aec0 10 manifest: total_size = 3299 2019-12-09T09:42:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.933+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.86 2019-12-09T09:42:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.933+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.933+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:46.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:46.933+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 2019-12-09T09:42:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.052+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6]) start _multipart_[] num_entries 1001 2019-12-09T09:42:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.052+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:47.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.130+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:47.173 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.185+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x56428a726360 s->prefetch_data=0 2019-12-09T09:42:47.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.185+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.195+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x16 2019-12-09T09:42:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.195+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T09:42:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.195+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:47.184 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.195+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:47.185 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.195+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:47.185 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.195+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2019-12-09T09:42:47.185 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.195+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=184 2019-12-09T09:42:47.204 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.216+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584972 cookie 94843789933584 notifier 4502 bl.length()=184 2019-12-09T09:42:47.771 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.783+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6 from cache 2019-12-09T09:42:47.772 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.783+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=241 2019-12-09T09:42:47.788 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:47.800+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584973 cookie 94843789934672 notifier 4502 bl.length()=241 2019-12-09T09:42:48.247 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.258+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x56428a7236c0 s->prefetch_data=0 2019-12-09T09:42:48.247 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.258+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2019-12-09T09:42:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.268+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x6 2019-12-09T09:42:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.268+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T09:42:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.268+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.268+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.268+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.268+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.280+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:42:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.280+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x1 2019-12-09T09:42:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.280+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T09:42:48.269 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.280+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 state=0x56428a726360 s->prefetch_data=0 2019-12-09T09:42:48.270 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.280+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 : miss 2019-12-09T09:42:48.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 info.flags=0x16 2019-12-09T09:42:48.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 to cache LRU end 2019-12-09T09:42:48.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:48.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:48.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:48.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:48.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:48.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:48.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.298+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:48.307 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.318+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:42:48.307 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.318+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 info.flags=0x13 2019-12-09T09:42:48.307 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.318+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 to cache LRU end 2019-12-09T09:42:48.307 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.318+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:48.307 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.318+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:48.308 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.318+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 2019-12-09T09:42:48.332 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.343+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10]) start [] num_entries 1100 2019-12-09T09:42:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.395+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T09:42:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.395+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T09:42:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.395+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T09:42:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.395+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T09:42:48.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.395+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.413+0000 7fe5f9a1aec0 10 manifest: total_size = 110 2019-12-09T09:42:48.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.413+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.305 2019-12-09T09:42:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.413+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.413+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.413+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 2019-12-09T09:42:48.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.539+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.547+0000 7fe5f9a1aec0 10 manifest: total_size = 4134 2019-12-09T09:42:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.547+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.307 2019-12-09T09:42:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.547+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.547+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.547+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 2019-12-09T09:42:48.686 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.698+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.700 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.711+0000 7fe5f9a1aec0 10 manifest: total_size = 4134 2019-12-09T09:42:48.700 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.711+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.306 2019-12-09T09:42:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.711+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.711+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.711+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 2019-12-09T09:42:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.865+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.874+0000 7fe5f9a1aec0 10 manifest: total_size = 49187 2019-12-09T09:42:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.874+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.304 2019-12-09T09:42:48.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.874+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.874+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:48.874+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 2019-12-09T09:42:49.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.016+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10]) start _multipart_[] num_entries 1001 2019-12-09T09:42:49.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.016+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:49.068 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.079+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.134+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x56428a725060 s->prefetch_data=0 2019-12-09T09:42:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.134+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.143+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x16 2019-12-09T09:42:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.143+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T09:42:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.143+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.143+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.143+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.143+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2019-12-09T09:42:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.143+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T09:42:49.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.153+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584963 cookie 94843789929744 notifier 4502 bl.length()=181 2019-12-09T09:42:49.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.751+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.10 from cache 2019-12-09T09:42:49.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.751+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=239 2019-12-09T09:42:49.762 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:49.773+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 94843789927760 notifier 4502 bl.length()=239 2019-12-09T09:42:50.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.138+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x56428a727900 s->prefetch_data=0 2019-12-09T09:42:50.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.138+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2019-12-09T09:42:50.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.153+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x6 2019-12-09T09:42:50.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.153+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T09:42:50.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.153+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:50.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.153+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:50.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.153+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:50.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.153+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.177+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:42:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.177+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x1 2019-12-09T09:42:50.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.177+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T09:42:50.167 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.177+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 state=0x56428a727650 s->prefetch_data=0 2019-12-09T09:42:50.167 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.177+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 : miss 2019-12-09T09:42:50.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 info.flags=0x16 2019-12-09T09:42:50.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 to cache LRU end 2019-12-09T09:42:50.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:50.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:50.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:50.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:50.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:50.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:50.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.195+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.228+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:42:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.228+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 info.flags=0x13 2019-12-09T09:42:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.228+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 to cache LRU end 2019-12-09T09:42:50.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.228+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.228+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:50.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.228+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 2019-12-09T09:42:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.244+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.2[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11]) start [] num_entries 1100 2019-12-09T09:42:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.269+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T09:42:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.269+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T09:42:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.269+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T09:42:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.269+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T09:42:50.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.269+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.282+0000 7fe5f9a1aec0 10 manifest: total_size = 527 2019-12-09T09:42:50.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.282+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.312 2019-12-09T09:42:50.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.282+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.282+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:50.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.282+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 2019-12-09T09:42:50.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.413+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.443+0000 7fe5f9a1aec0 10 manifest: total_size = 524288 2019-12-09T09:42:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.443+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.314 2019-12-09T09:42:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.443+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.443+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:50.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.443+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 2019-12-09T09:42:50.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.567+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.615+0000 7fe5f9a1aec0 10 manifest: total_size = 524288 2019-12-09T09:42:50.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.615+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.313 2019-12-09T09:42:50.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.615+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.615+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:50.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.615+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 2019-12-09T09:42:50.702 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.713+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.725+0000 7fe5f9a1aec0 10 manifest: total_size = 62 2019-12-09T09:42:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.725+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.311 2019-12-09T09:42:50.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.725+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x56428a722ca8 s->prefetch_data=0 2019-12-09T09:42:50.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.725+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a722ca8 2019-12-09T09:42:50.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.725+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 2019-12-09T09:42:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.902+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.2[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11]) start _multipart_[] num_entries 1001 2019-12-09T09:42:50.891 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.902+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:50.965+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:51.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.042+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x56428a723990 s->prefetch_data=0 2019-12-09T09:42:51.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.042+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.065+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x16 2019-12-09T09:42:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.065+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T09:42:51.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.065+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.065+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:51.055 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.065+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.065+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2019-12-09T09:42:51.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.065+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T09:42:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.083+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 94843789929744 notifier 4502 bl.length()=181 2019-12-09T09:42:51.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.788+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.11 from cache 2019-12-09T09:42:51.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.788+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=239 2019-12-09T09:42:51.788 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:51.800+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94843789927760 notifier 4502 bl.length()=239 2019-12-09T09:42:52.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.434+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x56428a728350 s->prefetch_data=0 2019-12-09T09:42:52.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.434+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2019-12-09T09:42:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.449+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x6 2019-12-09T09:42:52.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.449+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T09:42:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.449+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.449+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.449+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x1, cached=0x6) 2019-12-09T09:42:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.449+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:52.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.468+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T09:42:52.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.468+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x1 2019-12-09T09:42:52.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.468+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T09:42:52.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.468+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ffc80 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 state=0x56428a727900 s->prefetch_data=0 2019-12-09T09:42:52.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.468+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 : miss 2019-12-09T09:42:52.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 info.flags=0x16 2019-12-09T09:42:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 to cache LRU end 2019-12-09T09:42:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:52.475 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:52.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.acl 2019-12-09T09:42:52.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 20 Read xattr: user.rgw.idtag 2019-12-09T09:42:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 : type miss (requested=0x13, cached=0x16) 2019-12-09T09:42:52.484 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.486+0000 7fe5f9a1aec0 20 rados->read ofs=0 len=0 2019-12-09T09:42:52.495 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.506+0000 7fe5f9a1aec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T09:42:52.495 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.507+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 info.flags=0x13 2019-12-09T09:42:52.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.507+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 to cache LRU end 2019-12-09T09:42:52.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.507+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:52.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.507+0000 7fe5f9a1aec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T09:42:52.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.507+0000 7fe5f9a1aec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 2019-12-09T09:42:52.520 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.532+0000 7fe5f9a1aec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.3[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12]) start [] num_entries 1100 2019-12-09T09:42:52.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.569+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T09:42:52.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.569+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T09:42:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.569+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T09:42:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.569+0000 7fe5f9a1aec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T09:42:52.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.569+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.581+0000 7fe5f9a1aec0 10 manifest: total_size = 0 2019-12-09T09:42:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.581+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.319 2019-12-09T09:42:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.581+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:52.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.581+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:52.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.581+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 2019-12-09T09:42:52.734 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.745+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:52.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.840+0000 7fe5f9a1aec0 10 manifest: total_size = 0 2019-12-09T09:42:52.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.840+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.321 2019-12-09T09:42:52.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.840+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:52.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.840+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:52.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.840+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 2019-12-09T09:42:52.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.982+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.993+0000 7fe5f9a1aec0 10 manifest: total_size = 0 2019-12-09T09:42:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.993+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.320 2019-12-09T09:42:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.993+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:52.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.993+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:52.993+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 2019-12-09T09:42:53.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.139+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:53.140 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.151+0000 7fe5f9a1aec0 10 manifest: total_size = 0 2019-12-09T09:42:53.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.151+0000 7fe5f9a1aec0 20 get_obj_state: setting s->obj_tag to d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.318 2019-12-09T09:42:53.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.151+0000 7fe5f9a1aec0 20 get_obj_state: rctx=0x7ffd7f6ffca0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x56428a725ca8 s->prefetch_data=0 2019-12-09T09:42:53.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.151+0000 7fe5f9a1aec0 20 prepare_atomic_modification: state is not atomic. state=0x56428a725ca8 2019-12-09T09:42:53.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.151+0000 7fe5f9a1aec0 20 bucket index object: default.rgw.buckets.index:.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 2019-12-09T09:42:53.277 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.288+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.3[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12]) start _multipart_[] num_entries 1001 2019-12-09T09:42:53.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.288+0000 7fe5f9a1aec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T09:42:53.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.361+0000 7fe5f9a1aec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12])"; objs.size()=0; is_truncated=0 2019-12-09T09:42:53.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.434+0000 7fe5f9a1aec0 20 get_system_obj_state: rctx=0x7ffd7f6ff638 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x56428a728350 s->prefetch_data=0 2019-12-09T09:42:53.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.434+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x16, cached=0x3) 2019-12-09T09:42:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.459+0000 7fe5f9a1aec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x16 2019-12-09T09:42:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.459+0000 7fe5f9a1aec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T09:42:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.459+0000 7fe5f9a1aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T09:42:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.459+0000 7fe5f9a1aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T09:42:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.459+0000 7fe5f9a1aec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : hit (requested=0x11, cached=0x17) 2019-12-09T09:42:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.459+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2019-12-09T09:42:53.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.459+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T09:42:53.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.471+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94843789929744 notifier 4502 bl.length()=181 2019-12-09T09:42:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.586+0000 7fe5f9a1aec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.12 from cache 2019-12-09T09:42:53.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.586+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=239 2019-12-09T09:42:53.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.597+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94843789927760 notifier 4502 bl.length()=239 2019-12-09T09:42:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.883+0000 7fe5f9a1aec0 10 removing key index: KWPORNRXZWIKYLCKKRHM 2019-12-09T09:42:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.883+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=170 2019-12-09T09:42:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:53.901+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584967 cookie 94843789943856 notifier 4502 bl.length()=170 2019-12-09T09:42:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.135+0000 7fe5f9a1aec0 10 removing email index: ragweed.client.0+test@test.test 2019-12-09T09:42:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.135+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=182 2019-12-09T09:42:54.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.146+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584974 cookie 94843789933584 notifier 4502 bl.length()=182 2019-12-09T09:42:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.511+0000 7fe5f9a1aec0 10 removing user buckets index 2019-12-09T09:42:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.511+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=173 2019-12-09T09:42:54.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.526+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94843789940208 notifier 4502 bl.length()=173 2019-12-09T09:42:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.937+0000 7fe5f9a1aec0 10 removing user index: ragweed.client.0 2019-12-09T09:42:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.937+0000 7fe5f9a1aec0 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2019-12-09T09:42:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.937+0000 7fe5f9a1aec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=165 2019-12-09T09:42:54.946 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:54.956+0000 7fe5c7fff700 10 RGWWatcher::handle_notify() notify_id 579820584975 cookie 94843789934672 notifier 4502 bl.length()=165 2019-12-09T09:42:55.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:55.758+0000 7fe5f9a1aec0 20 remove_watcher() i=0 2019-12-09T09:42:55.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:55.758+0000 7fe5f9a1aec0 2 removed watcher, disabling cache 2019-12-09T09:42:55.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:55.823+0000 7fe5f9a1aec0 20 remove_watcher() i=1 2019-12-09T09:42:55.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:55.897+0000 7fe5f9a1aec0 20 remove_watcher() i=2 2019-12-09T09:42:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:55.943+0000 7fe5f9a1aec0 20 remove_watcher() i=3 2019-12-09T09:42:55.987 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:55.998+0000 7fe5f9a1aec0 20 remove_watcher() i=4 2019-12-09T09:42:56.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:56.038+0000 7fe5f9a1aec0 20 remove_watcher() i=5 2019-12-09T09:42:56.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:56.066+0000 7fe5f9a1aec0 20 remove_watcher() i=6 2019-12-09T09:42:56.089 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:42:56.100+0000 7fe5f9a1aec0 20 remove_watcher() i=7 2019-12-09T09:42:56.096 INFO:tasks.ragweed:Removing ragweed... 2019-12-09T09:42:56.096 INFO:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/ragweed 2019-12-09T09:42:56.181 INFO:teuthology.run_tasks:Running task s3tests-java... 2019-12-09T09:42:56.225 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2019-12-09T09:42:56.226 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2019-12-09T09:42:56.226 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2019-12-09T09:42:56.226 INFO:teuthology.orchestra.run.smithi033:> git clone -b master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2019-12-09T09:42:56.271 INFO:teuthology.orchestra.run.smithi033.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2019-12-09T09:43:00.147 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2019-12-09T09:43:00.147 INFO:teuthology.orchestra.run.smithi033:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2019-12-09T09:43:02.498 INFO:teuthology.orchestra.run.smithi033.stdout: Gradle will be installed under /opt/gradle 2019-12-09T09:43:02.539 INFO:teuthology.orchestra.run.smithi033.stderr:--2019-12-09 09:43:02-- https://services.gradle.org/distributions/gradle-4.7-bin.zip 2019-12-09T09:43:02.550 INFO:teuthology.orchestra.run.smithi033.stderr:Resolving services.gradle.org (services.gradle.org)... 104.18.190.9, 104.18.191.9, 2606:4700::6812:bf09, ... 2019-12-09T09:43:02.558 INFO:teuthology.orchestra.run.smithi033.stderr:Connecting to services.gradle.org (services.gradle.org)|104.18.190.9|:443... connected. 2019-12-09T09:43:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2019-12-09T09:43:02.599 INFO:teuthology.orchestra.run.smithi033.stderr:Location: https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip [following] 2019-12-09T09:43:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:--2019-12-09 09:43:02-- https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip 2019-12-09T09:43:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:Resolving downloads.gradle-dn.com (downloads.gradle-dn.com)... 104.17.160.20, 104.17.159.20, 2606:4700::6811:a014, ... 2019-12-09T09:43:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:Connecting to downloads.gradle-dn.com (downloads.gradle-dn.com)|104.17.160.20|:443... connected. 2019-12-09T09:43:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:HTTP request sent, awaiting response... 200 OK 2019-12-09T09:43:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:Length: 75323957 (72M) [application/zip] 2019-12-09T09:43:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:Saving to: ‘gradle-4.7-bin.zip’ 2019-12-09T09:43:02.645 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T09:43:02.658 INFO:teuthology.orchestra.run.smithi033.stderr: 0K .......... .......... .......... .......... .......... 0% 6.46M 11s 2019-12-09T09:43:02.658 INFO:teuthology.orchestra.run.smithi033.stderr: 50K .......... .......... .......... .......... .......... 0% 116M 6s 2019-12-09T09:43:02.669 INFO:teuthology.orchestra.run.smithi033.stderr: 100K .......... .......... .......... .......... .......... 0% 7.34M 7s 2019-12-09T09:43:02.669 INFO:teuthology.orchestra.run.smithi033.stderr: 150K .......... .......... .......... .......... .......... 0% 42.8M 6s 2019-12-09T09:43:02.669 INFO:teuthology.orchestra.run.smithi033.stderr: 200K .......... .......... .......... .......... .......... 0% 105M 5s 2019-12-09T09:43:02.670 INFO:teuthology.orchestra.run.smithi033.stderr: 250K .......... .......... .......... .......... .......... 0% 131M 4s 2019-12-09T09:43:02.670 INFO:teuthology.orchestra.run.smithi033.stderr: 300K .......... .......... .......... .......... .......... 0% 146M 4s 2019-12-09T09:43:02.670 INFO:teuthology.orchestra.run.smithi033.stderr: 350K .......... .......... .......... .......... .......... 0% 8.55M 4s 2019-12-09T09:43:02.670 INFO:teuthology.orchestra.run.smithi033.stderr: 400K .......... .......... .......... .......... .......... 0% 125M 4s 2019-12-09T09:43:02.688 INFO:teuthology.orchestra.run.smithi033.stderr: 450K .......... .......... .......... .......... .......... 0% 139M 3s 2019-12-09T09:43:02.688 INFO:teuthology.orchestra.run.smithi033.stderr: 500K .......... .......... .......... .......... .......... 0% 129M 3s 2019-12-09T09:43:02.688 INFO:teuthology.orchestra.run.smithi033.stderr: 550K .......... .......... .......... .......... .......... 0% 105M 3s 2019-12-09T09:43:02.689 INFO:teuthology.orchestra.run.smithi033.stderr: 600K .......... .......... .......... .......... .......... 0% 132M 3s 2019-12-09T09:43:02.689 INFO:teuthology.orchestra.run.smithi033.stderr: 650K .......... .......... .......... .......... .......... 0% 174M 3s 2019-12-09T09:43:02.689 INFO:teuthology.orchestra.run.smithi033.stderr: 700K .......... .......... .......... .......... .......... 1% 9.86M 3s 2019-12-09T09:43:02.689 INFO:teuthology.orchestra.run.smithi033.stderr: 750K .......... .......... .......... .......... .......... 1% 78.6M 3s 2019-12-09T09:43:02.689 INFO:teuthology.orchestra.run.smithi033.stderr: 800K .......... .......... .......... .......... .......... 1% 156M 3s 2019-12-09T09:43:02.689 INFO:teuthology.orchestra.run.smithi033.stderr: 850K .......... .......... .......... .......... .......... 1% 156M 3s 2019-12-09T09:43:02.689 INFO:teuthology.orchestra.run.smithi033.stderr: 900K .......... .......... .......... .......... .......... 1% 80.5M 2s 2019-12-09T09:43:02.690 INFO:teuthology.orchestra.run.smithi033.stderr: 950K .......... .......... .......... .......... .......... 1% 188M 2s 2019-12-09T09:43:02.690 INFO:teuthology.orchestra.run.smithi033.stderr: 1000K .......... .......... .......... .......... .......... 1% 109M 2s 2019-12-09T09:43:02.690 INFO:teuthology.orchestra.run.smithi033.stderr: 1050K .......... .......... .......... .......... .......... 1% 137M 2s 2019-12-09T09:43:02.690 INFO:teuthology.orchestra.run.smithi033.stderr: 1100K .......... .......... .......... .......... .......... 1% 145M 2s 2019-12-09T09:43:02.690 INFO:teuthology.orchestra.run.smithi033.stderr: 1150K .......... .......... .......... .......... .......... 1% 95.3M 2s 2019-12-09T09:43:02.690 INFO:teuthology.orchestra.run.smithi033.stderr: 1200K .......... .......... .......... .......... .......... 1% 160M 2s 2019-12-09T09:43:02.690 INFO:teuthology.orchestra.run.smithi033.stderr: 1250K .......... .......... .......... .......... .......... 1% 128M 2s 2019-12-09T09:43:02.691 INFO:teuthology.orchestra.run.smithi033.stderr: 1300K .......... .......... .......... .......... .......... 1% 151M 2s 2019-12-09T09:43:03.913 INFO:teuthology.orchestra.run.smithi033.stderr: 1350K .......... .......... .......... .......... .......... 1% 122M 2s 2019-12-09T09:43:03.913 INFO:teuthology.orchestra.run.smithi033.stderr: 1400K .......... .......... .......... .......... .......... 1% 131M 2s 2019-12-09T09:43:03.913 INFO:teuthology.orchestra.run.smithi033.stderr: 1450K .......... .......... .......... .......... .......... 2% 21.7M 2s 2019-12-09T09:43:03.913 INFO:teuthology.orchestra.run.smithi033.stderr: 1500K .......... .......... .......... .......... .......... 2% 148M 2s 2019-12-09T09:43:03.914 INFO:teuthology.orchestra.run.smithi033.stderr: 1550K .......... .......... .......... .......... .......... 2% 118M 2s 2019-12-09T09:43:03.914 INFO:teuthology.orchestra.run.smithi033.stderr: 1600K .......... .......... .......... .......... .......... 2% 138M 2s 2019-12-09T09:43:03.914 INFO:teuthology.orchestra.run.smithi033.stderr: 1650K .......... .......... .......... .......... .......... 2% 133M 2s 2019-12-09T09:43:03.914 INFO:teuthology.orchestra.run.smithi033.stderr: 1700K .......... .......... .......... .......... .......... 2% 147M 2s 2019-12-09T09:43:03.914 INFO:teuthology.orchestra.run.smithi033.stderr: 1750K .......... .......... .......... .......... .......... 2% 150M 2s 2019-12-09T09:43:03.914 INFO:teuthology.orchestra.run.smithi033.stderr: 1800K .......... .......... .......... .......... .......... 2% 209M 2s 2019-12-09T09:43:03.914 INFO:teuthology.orchestra.run.smithi033.stderr: 1850K .......... .......... .......... .......... .......... 2% 216M 2s 2019-12-09T09:43:03.915 INFO:teuthology.orchestra.run.smithi033.stderr: 1900K .......... .......... .......... .......... .......... 2% 258M 1s 2019-12-09T09:43:03.915 INFO:teuthology.orchestra.run.smithi033.stderr: 1950K .......... .......... .......... .......... .......... 2% 171M 1s 2019-12-09T09:43:03.915 INFO:teuthology.orchestra.run.smithi033.stderr: 2000K .......... .......... .......... .......... .......... 2% 190M 1s 2019-12-09T09:43:03.915 INFO:teuthology.orchestra.run.smithi033.stderr: 2050K .......... .......... .......... .......... .......... 2% 152M 1s 2019-12-09T09:43:03.915 INFO:teuthology.orchestra.run.smithi033.stderr: 2100K .......... .......... .......... .......... .......... 2% 158M 1s 2019-12-09T09:43:03.915 INFO:teuthology.orchestra.run.smithi033.stderr: 2150K .......... .......... .......... .......... .......... 2% 202M 1s 2019-12-09T09:43:03.915 INFO:teuthology.orchestra.run.smithi033.stderr: 2200K .......... .......... .......... .......... .......... 3% 148M 1s 2019-12-09T09:43:03.916 INFO:teuthology.orchestra.run.smithi033.stderr: 2250K .......... .......... .......... .......... .......... 3% 199M 1s 2019-12-09T09:43:03.916 INFO:teuthology.orchestra.run.smithi033.stderr: 2300K .......... .......... .......... .......... .......... 3% 258M 1s 2019-12-09T09:43:03.916 INFO:teuthology.orchestra.run.smithi033.stderr: 2350K .......... .......... .......... .......... .......... 3% 175M 1s 2019-12-09T09:43:03.916 INFO:teuthology.orchestra.run.smithi033.stderr: 2400K .......... .......... .......... .......... .......... 3% 141M 1s 2019-12-09T09:43:03.916 INFO:teuthology.orchestra.run.smithi033.stderr: 2450K .......... .......... .......... .......... .......... 3% 227M 1s 2019-12-09T09:43:03.916 INFO:teuthology.orchestra.run.smithi033.stderr: 2500K .......... .......... .......... .......... .......... 3% 230M 1s 2019-12-09T09:43:03.916 INFO:teuthology.orchestra.run.smithi033.stderr: 2550K .......... .......... .......... .......... .......... 3% 190M 1s 2019-12-09T09:43:03.917 INFO:teuthology.orchestra.run.smithi033.stderr: 2600K .......... .......... .......... .......... .......... 3% 195M 1s 2019-12-09T09:43:03.917 INFO:teuthology.orchestra.run.smithi033.stderr: 2650K .......... .......... .......... .......... .......... 3% 233M 1s 2019-12-09T09:43:03.917 INFO:teuthology.orchestra.run.smithi033.stderr: 2700K .......... .......... .......... .......... .......... 3% 171M 1s 2019-12-09T09:43:03.917 INFO:teuthology.orchestra.run.smithi033.stderr: 2750K .......... .......... .......... .......... .......... 3% 150M 1s 2019-12-09T09:43:03.917 INFO:teuthology.orchestra.run.smithi033.stderr: 2800K .......... .......... .......... .......... .......... 3% 233M 1s 2019-12-09T09:43:03.917 INFO:teuthology.orchestra.run.smithi033.stderr: 2850K .......... .......... .......... .......... .......... 3% 246M 1s 2019-12-09T09:43:03.917 INFO:teuthology.orchestra.run.smithi033.stderr: 2900K .......... .......... .......... .......... .......... 4% 226M 1s 2019-12-09T09:43:03.918 INFO:teuthology.orchestra.run.smithi033.stderr: 2950K .......... .......... .......... .......... .......... 4% 72.3M 1s 2019-12-09T09:43:03.918 INFO:teuthology.orchestra.run.smithi033.stderr: 3000K .......... .......... .......... .......... .......... 4% 245M 1s 2019-12-09T09:43:03.918 INFO:teuthology.orchestra.run.smithi033.stderr: 3050K .......... .......... .......... .......... .......... 4% 142M 1s 2019-12-09T09:43:03.918 INFO:teuthology.orchestra.run.smithi033.stderr: 3100K .......... .......... .......... .......... .......... 4% 142M 1s 2019-12-09T09:43:03.918 INFO:teuthology.orchestra.run.smithi033.stderr: 3150K .......... .......... .......... .......... .......... 4% 118M 1s 2019-12-09T09:43:03.919 INFO:teuthology.orchestra.run.smithi033.stderr: 3200K .......... .......... .......... .......... .......... 4% 229M 1s 2019-12-09T09:43:03.919 INFO:teuthology.orchestra.run.smithi033.stderr: 3250K .......... .......... .......... .......... .......... 4% 246M 1s 2019-12-09T09:43:03.919 INFO:teuthology.orchestra.run.smithi033.stderr: 3300K .......... .......... .......... .......... .......... 4% 225M 1s 2019-12-09T09:43:03.919 INFO:teuthology.orchestra.run.smithi033.stderr: 3350K .......... .......... .......... .......... .......... 4% 212M 1s 2019-12-09T09:43:03.919 INFO:teuthology.orchestra.run.smithi033.stderr: 3400K .......... .......... .......... .......... .......... 4% 228M 1s 2019-12-09T09:43:03.919 INFO:teuthology.orchestra.run.smithi033.stderr: 3450K .......... .......... .......... .......... .......... 4% 204M 1s 2019-12-09T09:43:03.919 INFO:teuthology.orchestra.run.smithi033.stderr: 3500K .......... .......... .......... .......... .......... 4% 228M 1s 2019-12-09T09:43:03.920 INFO:teuthology.orchestra.run.smithi033.stderr: 3550K .......... .......... .......... .......... .......... 4% 192M 1s 2019-12-09T09:43:03.920 INFO:teuthology.orchestra.run.smithi033.stderr: 3600K .......... .......... .......... .......... .......... 4% 235M 1s 2019-12-09T09:43:03.920 INFO:teuthology.orchestra.run.smithi033.stderr: 3650K .......... .......... .......... .......... .......... 5% 228M 1s 2019-12-09T09:43:03.920 INFO:teuthology.orchestra.run.smithi033.stderr: 3700K .......... .......... .......... .......... .......... 5% 229M 1s 2019-12-09T09:43:03.920 INFO:teuthology.orchestra.run.smithi033.stderr: 3750K .......... .......... .......... .......... .......... 5% 216M 1s 2019-12-09T09:43:03.920 INFO:teuthology.orchestra.run.smithi033.stderr: 3800K .......... .......... .......... .......... .......... 5% 237M 1s 2019-12-09T09:43:03.920 INFO:teuthology.orchestra.run.smithi033.stderr: 3850K .......... .......... .......... .......... .......... 5% 235M 1s 2019-12-09T09:43:03.921 INFO:teuthology.orchestra.run.smithi033.stderr: 3900K .......... .......... .......... .......... .......... 5% 238M 1s 2019-12-09T09:43:03.921 INFO:teuthology.orchestra.run.smithi033.stderr: 3950K .......... .......... .......... .......... .......... 5% 197M 1s 2019-12-09T09:43:03.921 INFO:teuthology.orchestra.run.smithi033.stderr: 4000K .......... .......... .......... .......... .......... 5% 230M 1s 2019-12-09T09:43:03.921 INFO:teuthology.orchestra.run.smithi033.stderr: 4050K .......... .......... .......... .......... .......... 5% 248M 1s 2019-12-09T09:43:03.921 INFO:teuthology.orchestra.run.smithi033.stderr: 4100K .......... .......... .......... .......... .......... 5% 230M 1s 2019-12-09T09:43:03.922 INFO:teuthology.orchestra.run.smithi033.stderr: 4150K .......... .......... .......... .......... .......... 5% 221M 1s 2019-12-09T09:43:03.922 INFO:teuthology.orchestra.run.smithi033.stderr: 4200K .......... .......... .......... .......... .......... 5% 235M 1s 2019-12-09T09:43:03.922 INFO:teuthology.orchestra.run.smithi033.stderr: 4250K .......... .......... .......... .......... .......... 5% 250M 1s 2019-12-09T09:43:03.922 INFO:teuthology.orchestra.run.smithi033.stderr: 4300K .......... .......... .......... .......... .......... 5% 168M 1s 2019-12-09T09:43:03.922 INFO:teuthology.orchestra.run.smithi033.stderr: 4350K .......... .......... .......... .......... .......... 5% 153M 1s 2019-12-09T09:43:03.922 INFO:teuthology.orchestra.run.smithi033.stderr: 4400K .......... .......... .......... .......... .......... 6% 197M 1s 2019-12-09T09:43:03.923 INFO:teuthology.orchestra.run.smithi033.stderr: 4450K .......... .......... .......... .......... .......... 6% 239M 1s 2019-12-09T09:43:03.923 INFO:teuthology.orchestra.run.smithi033.stderr: 4500K .......... .......... .......... .......... .......... 6% 245M 1s 2019-12-09T09:43:03.923 INFO:teuthology.orchestra.run.smithi033.stderr: 4550K .......... .......... .......... .......... .......... 6% 188M 1s 2019-12-09T09:43:03.923 INFO:teuthology.orchestra.run.smithi033.stderr: 4600K .......... .......... .......... .......... .......... 6% 209M 1s 2019-12-09T09:43:03.923 INFO:teuthology.orchestra.run.smithi033.stderr: 4650K .......... .......... .......... .......... .......... 6% 210M 1s 2019-12-09T09:43:03.923 INFO:teuthology.orchestra.run.smithi033.stderr: 4700K .......... .......... .......... .......... .......... 6% 237M 1s 2019-12-09T09:43:03.923 INFO:teuthology.orchestra.run.smithi033.stderr: 4750K .......... .......... .......... .......... .......... 6% 187M 1s 2019-12-09T09:43:03.924 INFO:teuthology.orchestra.run.smithi033.stderr: 4800K .......... .......... .......... .......... .......... 6% 243M 1s 2019-12-09T09:43:03.924 INFO:teuthology.orchestra.run.smithi033.stderr: 4850K .......... .......... .......... .......... .......... 6% 272M 1s 2019-12-09T09:43:03.924 INFO:teuthology.orchestra.run.smithi033.stderr: 4900K .......... .......... .......... .......... .......... 6% 247M 1s 2019-12-09T09:43:03.924 INFO:teuthology.orchestra.run.smithi033.stderr: 4950K .......... .......... .......... .......... .......... 6% 218M 1s 2019-12-09T09:43:03.924 INFO:teuthology.orchestra.run.smithi033.stderr: 5000K .......... .......... .......... .......... .......... 6% 228M 1s 2019-12-09T09:43:03.924 INFO:teuthology.orchestra.run.smithi033.stderr: 5050K .......... .......... .......... .......... .......... 6% 239M 1s 2019-12-09T09:43:03.925 INFO:teuthology.orchestra.run.smithi033.stderr: 5100K .......... .......... .......... .......... .......... 7% 258M 1s 2019-12-09T09:43:03.925 INFO:teuthology.orchestra.run.smithi033.stderr: 5150K .......... .......... .......... .......... .......... 7% 204M 1s 2019-12-09T09:43:03.925 INFO:teuthology.orchestra.run.smithi033.stderr: 5200K .......... .......... .......... .......... .......... 7% 240M 1s 2019-12-09T09:43:03.925 INFO:teuthology.orchestra.run.smithi033.stderr: 5250K .......... .......... .......... .......... .......... 7% 244M 1s 2019-12-09T09:43:03.925 INFO:teuthology.orchestra.run.smithi033.stderr: 5300K .......... .......... .......... .......... .......... 7% 276M 1s 2019-12-09T09:43:03.925 INFO:teuthology.orchestra.run.smithi033.stderr: 5350K .......... .......... .......... .......... .......... 7% 223M 1s 2019-12-09T09:43:03.925 INFO:teuthology.orchestra.run.smithi033.stderr: 5400K .......... .......... .......... .......... .......... 7% 236M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5450K .......... .......... .......... .......... .......... 7% 232M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5500K .......... .......... .......... .......... .......... 7% 245M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5550K .......... .......... .......... .......... .......... 7% 139M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5600K .......... .......... .......... .......... .......... 7% 160M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5650K .......... .......... .......... .......... .......... 7% 152M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5700K .......... .......... .......... .......... .......... 7% 161M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5750K .......... .......... .......... .......... .......... 7% 144M 1s 2019-12-09T09:43:03.926 INFO:teuthology.orchestra.run.smithi033.stderr: 5800K .......... .......... .......... .......... .......... 7% 157M 1s 2019-12-09T09:43:03.927 INFO:teuthology.orchestra.run.smithi033.stderr: 5850K .......... .......... .......... .......... .......... 8% 151M 1s 2019-12-09T09:43:03.927 INFO:teuthology.orchestra.run.smithi033.stderr: 5900K .......... .......... .......... .......... .......... 8% 147M 1s 2019-12-09T09:43:03.927 INFO:teuthology.orchestra.run.smithi033.stderr: 5950K .......... .......... .......... .......... .......... 8% 130M 1s 2019-12-09T09:43:03.927 INFO:teuthology.orchestra.run.smithi033.stderr: 6000K .......... .......... .......... .......... .......... 8% 161M 1s 2019-12-09T09:43:03.927 INFO:teuthology.orchestra.run.smithi033.stderr: 6050K .......... .......... .......... .......... .......... 8% 185M 1s 2019-12-09T09:43:03.927 INFO:teuthology.orchestra.run.smithi033.stderr: 6100K .......... .......... .......... .......... .......... 8% 145M 1s 2019-12-09T09:43:03.927 INFO:teuthology.orchestra.run.smithi033.stderr: 6150K .......... .......... .......... .......... .......... 8% 153M 1s 2019-12-09T09:43:03.928 INFO:teuthology.orchestra.run.smithi033.stderr: 6200K .......... .......... .......... .......... .......... 8% 184M 1s 2019-12-09T09:43:03.928 INFO:teuthology.orchestra.run.smithi033.stderr: 6250K .......... .......... .......... .......... .......... 8% 164M 1s 2019-12-09T09:43:03.928 INFO:teuthology.orchestra.run.smithi033.stderr: 6300K .......... .......... .......... .......... .......... 8% 165M 1s 2019-12-09T09:43:03.928 INFO:teuthology.orchestra.run.smithi033.stderr: 6350K .......... .......... .......... .......... .......... 8% 157M 1s 2019-12-09T09:43:03.928 INFO:teuthology.orchestra.run.smithi033.stderr: 6400K .......... .......... .......... .......... .......... 8% 186M 1s 2019-12-09T09:43:03.929 INFO:teuthology.orchestra.run.smithi033.stderr: 6450K .......... .......... .......... .......... .......... 8% 166M 1s 2019-12-09T09:43:03.929 INFO:teuthology.orchestra.run.smithi033.stderr: 6500K .......... .......... .......... .......... .......... 8% 188M 1s 2019-12-09T09:43:03.929 INFO:teuthology.orchestra.run.smithi033.stderr: 6550K .......... .......... .......... .......... .......... 8% 145M 1s 2019-12-09T09:43:03.929 INFO:teuthology.orchestra.run.smithi033.stderr: 6600K .......... .......... .......... .......... .......... 9% 180M 1s 2019-12-09T09:43:03.929 INFO:teuthology.orchestra.run.smithi033.stderr: 6650K .......... .......... .......... .......... .......... 9% 167M 1s 2019-12-09T09:43:03.930 INFO:teuthology.orchestra.run.smithi033.stderr: 6700K .......... .......... .......... .......... .......... 9% 193M 1s 2019-12-09T09:43:03.930 INFO:teuthology.orchestra.run.smithi033.stderr: 6750K .......... .......... .......... .......... .......... 9% 171M 1s 2019-12-09T09:43:03.930 INFO:teuthology.orchestra.run.smithi033.stderr: 6800K .......... .......... .......... .......... .......... 9% 179M 1s 2019-12-09T09:43:03.930 INFO:teuthology.orchestra.run.smithi033.stderr: 6850K .......... .......... .......... .......... .......... 9% 183M 1s 2019-12-09T09:43:03.930 INFO:teuthology.orchestra.run.smithi033.stderr: 6900K .......... .......... .......... .......... .......... 9% 185M 1s 2019-12-09T09:43:03.930 INFO:teuthology.orchestra.run.smithi033.stderr: 6950K .......... .......... .......... .......... .......... 9% 146M 1s 2019-12-09T09:43:03.930 INFO:teuthology.orchestra.run.smithi033.stderr: 7000K .......... .......... .......... .......... .......... 9% 176M 1s 2019-12-09T09:43:03.931 INFO:teuthology.orchestra.run.smithi033.stderr: 7050K .......... .......... .......... .......... .......... 9% 192M 1s 2019-12-09T09:43:03.931 INFO:teuthology.orchestra.run.smithi033.stderr: 7100K .......... .......... .......... .......... .......... 9% 177M 1s 2019-12-09T09:43:03.931 INFO:teuthology.orchestra.run.smithi033.stderr: 7150K .......... .......... .......... .......... .......... 9% 151M 1s 2019-12-09T09:43:03.931 INFO:teuthology.orchestra.run.smithi033.stderr: 7200K .......... .......... .......... .......... .......... 9% 177M 1s 2019-12-09T09:43:03.931 INFO:teuthology.orchestra.run.smithi033.stderr: 7250K .......... .......... .......... .......... .......... 9% 188M 1s 2019-12-09T09:43:03.931 INFO:teuthology.orchestra.run.smithi033.stderr: 7300K .......... .......... .......... .......... .......... 9% 162M 1s 2019-12-09T09:43:03.931 INFO:teuthology.orchestra.run.smithi033.stderr: 7350K .......... .......... .......... .......... .......... 10% 132M 1s 2019-12-09T09:43:03.932 INFO:teuthology.orchestra.run.smithi033.stderr: 7400K .......... .......... .......... .......... .......... 10% 193M 1s 2019-12-09T09:43:03.932 INFO:teuthology.orchestra.run.smithi033.stderr: 7450K .......... .......... .......... .......... .......... 10% 139M 1s 2019-12-09T09:43:03.932 INFO:teuthology.orchestra.run.smithi033.stderr: 7500K .......... .......... .......... .......... .......... 10% 183M 1s 2019-12-09T09:43:03.932 INFO:teuthology.orchestra.run.smithi033.stderr: 7550K .......... .......... .......... .......... .......... 10% 161M 1s 2019-12-09T09:43:03.932 INFO:teuthology.orchestra.run.smithi033.stderr: 7600K .......... .......... .......... .......... .......... 10% 175M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 7650K .......... .......... .......... .......... .......... 10% 172M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 7700K .......... .......... .......... .......... .......... 10% 156M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 7750K .......... .......... .......... .......... .......... 10% 137M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 7800K .......... .......... .......... .......... .......... 10% 170M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 7850K .......... .......... .......... .......... .......... 10% 178M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 7900K .......... .......... .......... .......... .......... 10% 199M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 7950K .......... .......... .......... .......... .......... 10% 173M 1s 2019-12-09T09:43:03.933 INFO:teuthology.orchestra.run.smithi033.stderr: 8000K .......... .......... .......... .......... .......... 10% 190M 1s 2019-12-09T09:43:03.934 INFO:teuthology.orchestra.run.smithi033.stderr: 8050K .......... .......... .......... .......... .......... 11% 193M 1s 2019-12-09T09:43:03.934 INFO:teuthology.orchestra.run.smithi033.stderr: 8100K .......... .......... .......... .......... .......... 11% 185M 1s 2019-12-09T09:43:03.934 INFO:teuthology.orchestra.run.smithi033.stderr: 8150K .......... .......... .......... .......... .......... 11% 158M 1s 2019-12-09T09:43:03.934 INFO:teuthology.orchestra.run.smithi033.stderr: 8200K .......... .......... .......... .......... .......... 11% 187M 1s 2019-12-09T09:43:03.934 INFO:teuthology.orchestra.run.smithi033.stderr: 8250K .......... .......... .......... .......... .......... 11% 198M 1s 2019-12-09T09:43:03.935 INFO:teuthology.orchestra.run.smithi033.stderr: 8300K .......... .......... .......... .......... .......... 11% 200M 1s 2019-12-09T09:43:03.935 INFO:teuthology.orchestra.run.smithi033.stderr: 8350K .......... .......... .......... .......... .......... 11% 168M 1s 2019-12-09T09:43:03.935 INFO:teuthology.orchestra.run.smithi033.stderr: 8400K .......... .......... .......... .......... .......... 11% 180M 1s 2019-12-09T09:43:03.935 INFO:teuthology.orchestra.run.smithi033.stderr: 8450K .......... .......... .......... .......... .......... 11% 184M 1s 2019-12-09T09:43:03.936 INFO:teuthology.orchestra.run.smithi033.stderr: 8500K .......... .......... .......... .......... .......... 11% 179M 1s 2019-12-09T09:43:03.936 INFO:teuthology.orchestra.run.smithi033.stderr: 8550K .......... .......... .......... .......... .......... 11% 151M 1s 2019-12-09T09:43:03.936 INFO:teuthology.orchestra.run.smithi033.stderr: 8600K .......... .......... .......... .......... .......... 11% 188M 1s 2019-12-09T09:43:03.936 INFO:teuthology.orchestra.run.smithi033.stderr: 8650K .......... .......... .......... .......... .......... 11% 173M 1s 2019-12-09T09:43:03.936 INFO:teuthology.orchestra.run.smithi033.stderr: 8700K .......... .......... .......... .......... .......... 11% 176M 1s 2019-12-09T09:43:03.936 INFO:teuthology.orchestra.run.smithi033.stderr: 8750K .......... .......... .......... .......... .......... 11% 159M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 8800K .......... .......... .......... .......... .......... 12% 167M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 8850K .......... .......... .......... .......... .......... 12% 181M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 8900K .......... .......... .......... .......... .......... 12% 162M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 8950K .......... .......... .......... .......... .......... 12% 116M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 9000K .......... .......... .......... .......... .......... 12% 169M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 9050K .......... .......... .......... .......... .......... 12% 151M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 9100K .......... .......... .......... .......... .......... 12% 173M 1s 2019-12-09T09:43:03.937 INFO:teuthology.orchestra.run.smithi033.stderr: 9150K .......... .......... .......... .......... .......... 12% 153M 1s 2019-12-09T09:43:03.938 INFO:teuthology.orchestra.run.smithi033.stderr: 9200K .......... .......... .......... .......... .......... 12% 187M 1s 2019-12-09T09:43:03.938 INFO:teuthology.orchestra.run.smithi033.stderr: 9250K .......... .......... .......... .......... .......... 12% 191M 1s 2019-12-09T09:43:03.938 INFO:teuthology.orchestra.run.smithi033.stderr: 9300K .......... .......... .......... .......... .......... 12% 191M 1s 2019-12-09T09:43:03.938 INFO:teuthology.orchestra.run.smithi033.stderr: 9350K .......... .......... .......... .......... .......... 12% 188M 1s 2019-12-09T09:43:03.938 INFO:teuthology.orchestra.run.smithi033.stderr: 9400K .......... .......... .......... .......... .......... 12% 197M 1s 2019-12-09T09:43:03.938 INFO:teuthology.orchestra.run.smithi033.stderr: 9450K .......... .......... .......... .......... .......... 12% 180M 1s 2019-12-09T09:43:03.938 INFO:teuthology.orchestra.run.smithi033.stderr: 9500K .......... .......... .......... .......... .......... 12% 193M 1s 2019-12-09T09:43:03.939 INFO:teuthology.orchestra.run.smithi033.stderr: 9550K .......... .......... .......... .......... .......... 13% 159M 1s 2019-12-09T09:43:03.939 INFO:teuthology.orchestra.run.smithi033.stderr: 9600K .......... .......... .......... .......... .......... 13% 204M 1s 2019-12-09T09:43:03.939 INFO:teuthology.orchestra.run.smithi033.stderr: 9650K .......... .......... .......... .......... .......... 13% 156M 1s 2019-12-09T09:43:03.939 INFO:teuthology.orchestra.run.smithi033.stderr: 9700K .......... .......... .......... .......... .......... 13% 182M 1s 2019-12-09T09:43:03.939 INFO:teuthology.orchestra.run.smithi033.stderr: 9750K .......... .......... .......... .......... .......... 13% 156M 1s 2019-12-09T09:43:03.939 INFO:teuthology.orchestra.run.smithi033.stderr: 9800K .......... .......... .......... .......... .......... 13% 184M 1s 2019-12-09T09:43:03.939 INFO:teuthology.orchestra.run.smithi033.stderr: 9850K .......... .......... .......... .......... .......... 13% 166M 1s 2019-12-09T09:43:03.940 INFO:teuthology.orchestra.run.smithi033.stderr: 9900K .......... .......... .......... .......... .......... 13% 210M 1s 2019-12-09T09:43:03.940 INFO:teuthology.orchestra.run.smithi033.stderr: 9950K .......... .......... .......... .......... .......... 13% 148M 1s 2019-12-09T09:43:03.940 INFO:teuthology.orchestra.run.smithi033.stderr: 10000K .......... .......... .......... .......... .......... 13% 143M 1s 2019-12-09T09:43:03.940 INFO:teuthology.orchestra.run.smithi033.stderr: 10050K .......... .......... .......... .......... .......... 13% 164M 1s 2019-12-09T09:43:03.940 INFO:teuthology.orchestra.run.smithi033.stderr: 10100K .......... .......... .......... .......... .......... 13% 203M 1s 2019-12-09T09:43:03.940 INFO:teuthology.orchestra.run.smithi033.stderr: 10150K .......... .......... .......... .......... .......... 13% 170M 1s 2019-12-09T09:43:03.940 INFO:teuthology.orchestra.run.smithi033.stderr: 10200K .......... .......... .......... .......... .......... 13% 187M 1s 2019-12-09T09:43:03.941 INFO:teuthology.orchestra.run.smithi033.stderr: 10250K .......... .......... .......... .......... .......... 14% 192M 1s 2019-12-09T09:43:03.941 INFO:teuthology.orchestra.run.smithi033.stderr: 10300K .......... .......... .......... .......... .......... 14% 191M 1s 2019-12-09T09:43:03.941 INFO:teuthology.orchestra.run.smithi033.stderr: 10350K .......... .......... .......... .......... .......... 14% 150M 1s 2019-12-09T09:43:03.941 INFO:teuthology.orchestra.run.smithi033.stderr: 10400K .......... .......... .......... .......... .......... 14% 208M 1s 2019-12-09T09:43:03.941 INFO:teuthology.orchestra.run.smithi033.stderr: 10450K .......... .......... .......... .......... .......... 14% 194M 1s 2019-12-09T09:43:03.941 INFO:teuthology.orchestra.run.smithi033.stderr: 10500K .......... .......... .......... .......... .......... 14% 137M 1s 2019-12-09T09:43:03.941 INFO:teuthology.orchestra.run.smithi033.stderr: 10550K .......... .......... .......... .......... .......... 14% 169M 1s 2019-12-09T09:43:03.942 INFO:teuthology.orchestra.run.smithi033.stderr: 10600K .......... .......... .......... .......... .......... 14% 191M 1s 2019-12-09T09:43:03.942 INFO:teuthology.orchestra.run.smithi033.stderr: 10650K .......... .......... .......... .......... .......... 14% 186M 1s 2019-12-09T09:43:03.942 INFO:teuthology.orchestra.run.smithi033.stderr: 10700K .......... .......... .......... .......... .......... 14% 198M 1s 2019-12-09T09:43:03.942 INFO:teuthology.orchestra.run.smithi033.stderr: 10750K .......... .......... .......... .......... .......... 14% 166M 1s 2019-12-09T09:43:03.942 INFO:teuthology.orchestra.run.smithi033.stderr: 10800K .......... .......... .......... .......... .......... 14% 209M 1s 2019-12-09T09:43:03.942 INFO:teuthology.orchestra.run.smithi033.stderr: 10850K .......... .......... .......... .......... .......... 14% 181M 1s 2019-12-09T09:43:03.942 INFO:teuthology.orchestra.run.smithi033.stderr: 10900K .......... .......... .......... .......... .......... 14% 193M 1s 2019-12-09T09:43:03.943 INFO:teuthology.orchestra.run.smithi033.stderr: 10950K .......... .......... .......... .......... .......... 14% 182M 1s 2019-12-09T09:43:03.943 INFO:teuthology.orchestra.run.smithi033.stderr: 11000K .......... .......... .......... .......... .......... 15% 196M 1s 2019-12-09T09:43:03.943 INFO:teuthology.orchestra.run.smithi033.stderr: 11050K .......... .......... .......... .......... .......... 15% 195M 1s 2019-12-09T09:43:03.943 INFO:teuthology.orchestra.run.smithi033.stderr: 11100K .......... .......... .......... .......... .......... 15% 196M 1s 2019-12-09T09:43:03.943 INFO:teuthology.orchestra.run.smithi033.stderr: 11150K .......... .......... .......... .......... .......... 15% 174M 1s 2019-12-09T09:43:03.943 INFO:teuthology.orchestra.run.smithi033.stderr: 11200K .......... .......... .......... .......... .......... 15% 211M 1s 2019-12-09T09:43:03.943 INFO:teuthology.orchestra.run.smithi033.stderr: 11250K .......... .......... .......... .......... .......... 15% 192M 0s 2019-12-09T09:43:03.944 INFO:teuthology.orchestra.run.smithi033.stderr: 11300K .......... .......... .......... .......... .......... 15% 212M 0s 2019-12-09T09:43:03.944 INFO:teuthology.orchestra.run.smithi033.stderr: 11350K .......... .......... .......... .......... .......... 15% 190M 0s 2019-12-09T09:43:03.944 INFO:teuthology.orchestra.run.smithi033.stderr: 11400K .......... .......... .......... .......... .......... 15% 199M 0s 2019-12-09T09:43:03.944 INFO:teuthology.orchestra.run.smithi033.stderr: 11450K .......... .......... .......... .......... .......... 15% 194M 0s 2019-12-09T09:43:03.944 INFO:teuthology.orchestra.run.smithi033.stderr: 11500K .......... .......... .......... .......... .......... 15% 169M 0s 2019-12-09T09:43:03.944 INFO:teuthology.orchestra.run.smithi033.stderr: 11550K .......... .......... .......... .......... .......... 15% 142M 0s 2019-12-09T09:43:03.944 INFO:teuthology.orchestra.run.smithi033.stderr: 11600K .......... .......... .......... .......... .......... 15% 159M 0s 2019-12-09T09:43:03.945 INFO:teuthology.orchestra.run.smithi033.stderr: 11650K .......... .......... .......... .......... .......... 15% 189M 0s 2019-12-09T09:43:03.945 INFO:teuthology.orchestra.run.smithi033.stderr: 11700K .......... .......... .......... .......... .......... 15% 197M 0s 2019-12-09T09:43:03.945 INFO:teuthology.orchestra.run.smithi033.stderr: 11750K .......... .......... .......... .......... .......... 16% 177M 0s 2019-12-09T09:43:03.945 INFO:teuthology.orchestra.run.smithi033.stderr: 11800K .......... .......... .......... .......... .......... 16% 191M 0s 2019-12-09T09:43:03.945 INFO:teuthology.orchestra.run.smithi033.stderr: 11850K .......... .......... .......... .......... .......... 16% 206M 0s 2019-12-09T09:43:03.945 INFO:teuthology.orchestra.run.smithi033.stderr: 11900K .......... .......... .......... .......... .......... 16% 142M 0s 2019-12-09T09:43:03.945 INFO:teuthology.orchestra.run.smithi033.stderr: 11950K .......... .......... .......... .......... .......... 16% 126M 0s 2019-12-09T09:43:03.946 INFO:teuthology.orchestra.run.smithi033.stderr: 12000K .......... .......... .......... .......... .......... 16% 172M 0s 2019-12-09T09:43:03.946 INFO:teuthology.orchestra.run.smithi033.stderr: 12050K .......... .......... .......... .......... .......... 16% 167M 0s 2019-12-09T09:43:03.946 INFO:teuthology.orchestra.run.smithi033.stderr: 12100K .......... .......... .......... .......... .......... 16% 210M 0s 2019-12-09T09:43:03.946 INFO:teuthology.orchestra.run.smithi033.stderr: 12150K .......... .......... .......... .......... .......... 16% 177M 0s 2019-12-09T09:43:03.946 INFO:teuthology.orchestra.run.smithi033.stderr: 12200K .......... .......... .......... .......... .......... 16% 214M 0s 2019-12-09T09:43:03.946 INFO:teuthology.orchestra.run.smithi033.stderr: 12250K .......... .......... .......... .......... .......... 16% 209M 0s 2019-12-09T09:43:03.946 INFO:teuthology.orchestra.run.smithi033.stderr: 12300K .......... .......... .......... .......... .......... 16% 200M 0s 2019-12-09T09:43:03.947 INFO:teuthology.orchestra.run.smithi033.stderr: 12350K .......... .......... .......... .......... .......... 16% 163M 0s 2019-12-09T09:43:03.947 INFO:teuthology.orchestra.run.smithi033.stderr: 12400K .......... .......... .......... .......... .......... 16% 190M 0s 2019-12-09T09:43:03.947 INFO:teuthology.orchestra.run.smithi033.stderr: 12450K .......... .......... .......... .......... .......... 16% 218M 0s 2019-12-09T09:43:03.947 INFO:teuthology.orchestra.run.smithi033.stderr: 12500K .......... .......... .......... .......... .......... 17% 190M 0s 2019-12-09T09:43:03.947 INFO:teuthology.orchestra.run.smithi033.stderr: 12550K .......... .......... .......... .......... .......... 17% 178M 0s 2019-12-09T09:43:03.947 INFO:teuthology.orchestra.run.smithi033.stderr: 12600K .......... .......... .......... .......... .......... 17% 200M 0s 2019-12-09T09:43:03.947 INFO:teuthology.orchestra.run.smithi033.stderr: 12650K .......... .......... .......... .......... .......... 17% 196M 0s 2019-12-09T09:43:03.948 INFO:teuthology.orchestra.run.smithi033.stderr: 12700K .......... .......... .......... .......... .......... 17% 204M 0s 2019-12-09T09:43:03.948 INFO:teuthology.orchestra.run.smithi033.stderr: 12750K .......... .......... .......... .......... .......... 17% 156M 0s 2019-12-09T09:43:03.948 INFO:teuthology.orchestra.run.smithi033.stderr: 12800K .......... .......... .......... .......... .......... 17% 199M 0s 2019-12-09T09:43:03.948 INFO:teuthology.orchestra.run.smithi033.stderr: 12850K .......... .......... .......... .......... .......... 17% 189M 0s 2019-12-09T09:43:03.948 INFO:teuthology.orchestra.run.smithi033.stderr: 12900K .......... .......... .......... .......... .......... 17% 217M 0s 2019-12-09T09:43:03.948 INFO:teuthology.orchestra.run.smithi033.stderr: 12950K .......... .......... .......... .......... .......... 17% 186M 0s 2019-12-09T09:43:03.948 INFO:teuthology.orchestra.run.smithi033.stderr: 13000K .......... .......... .......... .......... .......... 17% 219M 0s 2019-12-09T09:43:03.949 INFO:teuthology.orchestra.run.smithi033.stderr: 13050K .......... .......... .......... .......... .......... 17% 196M 0s 2019-12-09T09:43:03.949 INFO:teuthology.orchestra.run.smithi033.stderr: 13100K .......... .......... .......... .......... .......... 17% 207M 0s 2019-12-09T09:43:03.949 INFO:teuthology.orchestra.run.smithi033.stderr: 13150K .......... .......... .......... .......... .......... 17% 183M 0s 2019-12-09T09:43:03.949 INFO:teuthology.orchestra.run.smithi033.stderr: 13200K .......... .......... .......... .......... .......... 18% 282M 0s 2019-12-09T09:43:03.949 INFO:teuthology.orchestra.run.smithi033.stderr: 13250K .......... .......... .......... .......... .......... 18% 244M 0s 2019-12-09T09:43:03.949 INFO:teuthology.orchestra.run.smithi033.stderr: 13300K .......... .......... .......... .......... .......... 18% 295M 0s 2019-12-09T09:43:03.949 INFO:teuthology.orchestra.run.smithi033.stderr: 13350K .......... .......... .......... .......... .......... 18% 281M 0s 2019-12-09T09:43:03.950 INFO:teuthology.orchestra.run.smithi033.stderr: 13400K .......... .......... .......... .......... .......... 18% 222M 0s 2019-12-09T09:43:03.950 INFO:teuthology.orchestra.run.smithi033.stderr: 13450K .......... .......... .......... .......... .......... 18% 210M 0s 2019-12-09T09:43:03.950 INFO:teuthology.orchestra.run.smithi033.stderr: 13500K .......... .......... .......... .......... .......... 18% 239M 0s 2019-12-09T09:43:03.950 INFO:teuthology.orchestra.run.smithi033.stderr: 13550K .......... .......... .......... .......... .......... 18% 221M 0s 2019-12-09T09:43:03.950 INFO:teuthology.orchestra.run.smithi033.stderr: 13600K .......... .......... .......... .......... .......... 18% 230M 0s 2019-12-09T09:43:03.950 INFO:teuthology.orchestra.run.smithi033.stderr: 13650K .......... .......... .......... .......... .......... 18% 231M 0s 2019-12-09T09:43:03.950 INFO:teuthology.orchestra.run.smithi033.stderr: 13700K .......... .......... .......... .......... .......... 18% 304M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 13750K .......... .......... .......... .......... .......... 18% 223M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 13800K .......... .......... .......... .......... .......... 18% 254M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 13850K .......... .......... .......... .......... .......... 18% 284M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 13900K .......... .......... .......... .......... .......... 18% 276M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 13950K .......... .......... .......... .......... .......... 19% 216M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 14000K .......... .......... .......... .......... .......... 19% 291M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 14050K .......... .......... .......... .......... .......... 19% 284M 0s 2019-12-09T09:43:03.951 INFO:teuthology.orchestra.run.smithi033.stderr: 14100K .......... .......... .......... .......... .......... 19% 262M 0s 2019-12-09T09:43:03.952 INFO:teuthology.orchestra.run.smithi033.stderr: 14150K .......... .......... .......... .......... .......... 19% 284M 0s 2019-12-09T09:43:03.952 INFO:teuthology.orchestra.run.smithi033.stderr: 14200K .......... .......... .......... .......... .......... 19% 286M 0s 2019-12-09T09:43:03.952 INFO:teuthology.orchestra.run.smithi033.stderr: 14250K .......... .......... .......... .......... .......... 19% 280M 0s 2019-12-09T09:43:03.952 INFO:teuthology.orchestra.run.smithi033.stderr: 14300K .......... .......... .......... .......... .......... 19% 273M 0s 2019-12-09T09:43:03.952 INFO:teuthology.orchestra.run.smithi033.stderr: 14350K .......... .......... .......... .......... .......... 19% 220M 0s 2019-12-09T09:43:03.952 INFO:teuthology.orchestra.run.smithi033.stderr: 14400K .......... .......... .......... .......... .......... 19% 291M 0s 2019-12-09T09:43:03.952 INFO:teuthology.orchestra.run.smithi033.stderr: 14450K .......... .......... .......... .......... .......... 19% 247M 0s 2019-12-09T09:43:03.953 INFO:teuthology.orchestra.run.smithi033.stderr: 14500K .......... .......... .......... .......... .......... 19% 242M 0s 2019-12-09T09:43:03.953 INFO:teuthology.orchestra.run.smithi033.stderr: 14550K .......... .......... .......... .......... .......... 19% 255M 0s 2019-12-09T09:43:03.953 INFO:teuthology.orchestra.run.smithi033.stderr: 14600K .......... .......... .......... .......... .......... 19% 292M 0s 2019-12-09T09:43:03.953 INFO:teuthology.orchestra.run.smithi033.stderr: 14650K .......... .......... .......... .......... .......... 19% 232M 0s 2019-12-09T09:43:03.953 INFO:teuthology.orchestra.run.smithi033.stderr: 14700K .......... .......... .......... .......... .......... 20% 226M 0s 2019-12-09T09:43:03.953 INFO:teuthology.orchestra.run.smithi033.stderr: 14750K .......... .......... .......... .......... .......... 20% 159M 0s 2019-12-09T09:43:03.953 INFO:teuthology.orchestra.run.smithi033.stderr: 14800K .......... .......... .......... .......... .......... 20% 178M 0s 2019-12-09T09:43:03.954 INFO:teuthology.orchestra.run.smithi033.stderr: 14850K .......... .......... .......... .......... .......... 20% 175M 0s 2019-12-09T09:43:03.954 INFO:teuthology.orchestra.run.smithi033.stderr: 14900K .......... .......... .......... .......... .......... 20% 167M 0s 2019-12-09T09:43:03.954 INFO:teuthology.orchestra.run.smithi033.stderr: 14950K .......... .......... .......... .......... .......... 20% 172M 0s 2019-12-09T09:43:03.954 INFO:teuthology.orchestra.run.smithi033.stderr: 15000K .......... .......... .......... .......... .......... 20% 208M 0s 2019-12-09T09:43:03.954 INFO:teuthology.orchestra.run.smithi033.stderr: 15050K .......... .......... .......... .......... .......... 20% 217M 0s 2019-12-09T09:43:03.954 INFO:teuthology.orchestra.run.smithi033.stderr: 15100K .......... .......... .......... .......... .......... 20% 186M 0s 2019-12-09T09:43:03.954 INFO:teuthology.orchestra.run.smithi033.stderr: 15150K .......... .......... .......... .......... .......... 20% 164M 0s 2019-12-09T09:43:03.955 INFO:teuthology.orchestra.run.smithi033.stderr: 15200K .......... .......... .......... .......... .......... 20% 200M 0s 2019-12-09T09:43:03.955 INFO:teuthology.orchestra.run.smithi033.stderr: 15250K .......... .......... .......... .......... .......... 20% 210M 0s 2019-12-09T09:43:03.955 INFO:teuthology.orchestra.run.smithi033.stderr: 15300K .......... .......... .......... .......... .......... 20% 215M 0s 2019-12-09T09:43:03.955 INFO:teuthology.orchestra.run.smithi033.stderr: 15350K .......... .......... .......... .......... .......... 20% 188M 0s 2019-12-09T09:43:03.955 INFO:teuthology.orchestra.run.smithi033.stderr: 15400K .......... .......... .......... .......... .......... 21% 222M 0s 2019-12-09T09:43:03.956 INFO:teuthology.orchestra.run.smithi033.stderr: 15450K .......... .......... .......... .......... .......... 21% 214M 0s 2019-12-09T09:43:03.956 INFO:teuthology.orchestra.run.smithi033.stderr: 15500K .......... .......... .......... .......... .......... 21% 203M 0s 2019-12-09T09:43:03.956 INFO:teuthology.orchestra.run.smithi033.stderr: 15550K .......... .......... .......... .......... .......... 21% 157M 0s 2019-12-09T09:43:03.956 INFO:teuthology.orchestra.run.smithi033.stderr: 15600K .......... .......... .......... .......... .......... 21% 213M 0s 2019-12-09T09:43:03.956 INFO:teuthology.orchestra.run.smithi033.stderr: 15650K .......... .......... .......... .......... .......... 21% 185M 0s 2019-12-09T09:43:03.956 INFO:teuthology.orchestra.run.smithi033.stderr: 15700K .......... .......... .......... .......... .......... 21% 270M 0s 2019-12-09T09:43:03.956 INFO:teuthology.orchestra.run.smithi033.stderr: 15750K .......... .......... .......... .......... .......... 21% 228M 0s 2019-12-09T09:43:03.957 INFO:teuthology.orchestra.run.smithi033.stderr: 15800K .......... .......... .......... .......... .......... 21% 335M 0s 2019-12-09T09:43:03.957 INFO:teuthology.orchestra.run.smithi033.stderr: 15850K .......... .......... .......... .......... .......... 21% 293M 0s 2019-12-09T09:43:03.957 INFO:teuthology.orchestra.run.smithi033.stderr: 15900K .......... .......... .......... .......... .......... 21% 264M 0s 2019-12-09T09:43:03.957 INFO:teuthology.orchestra.run.smithi033.stderr: 15950K .......... .......... .......... .......... .......... 21% 222M 0s 2019-12-09T09:43:03.957 INFO:teuthology.orchestra.run.smithi033.stderr: 16000K .......... .......... .......... .......... .......... 21% 335M 0s 2019-12-09T09:43:03.957 INFO:teuthology.orchestra.run.smithi033.stderr: 16050K .......... .......... .......... .......... .......... 21% 295M 0s 2019-12-09T09:43:03.957 INFO:teuthology.orchestra.run.smithi033.stderr: 16100K .......... .......... .......... .......... .......... 21% 281M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16150K .......... .......... .......... .......... .......... 22% 238M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16200K .......... .......... .......... .......... .......... 22% 315M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16250K .......... .......... .......... .......... .......... 22% 282M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16300K .......... .......... .......... .......... .......... 22% 255M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16350K .......... .......... .......... .......... .......... 22% 278M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16400K .......... .......... .......... .......... .......... 22% 293M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16450K .......... .......... .......... .......... .......... 22% 306M 0s 2019-12-09T09:43:03.958 INFO:teuthology.orchestra.run.smithi033.stderr: 16500K .......... .......... .......... .......... .......... 22% 265M 0s 2019-12-09T09:43:03.959 INFO:teuthology.orchestra.run.smithi033.stderr: 16550K .......... .......... .......... .......... .......... 22% 281M 0s 2019-12-09T09:43:03.959 INFO:teuthology.orchestra.run.smithi033.stderr: 16600K .......... .......... .......... .......... .......... 22% 309M 0s 2019-12-09T09:43:03.959 INFO:teuthology.orchestra.run.smithi033.stderr: 16650K .......... .......... .......... .......... .......... 22% 296M 0s 2019-12-09T09:43:03.959 INFO:teuthology.orchestra.run.smithi033.stderr: 16700K .......... .......... .......... .......... .......... 22% 294M 0s 2019-12-09T09:43:03.959 INFO:teuthology.orchestra.run.smithi033.stderr: 16750K .......... .......... .......... .......... .......... 22% 198M 0s 2019-12-09T09:43:03.959 INFO:teuthology.orchestra.run.smithi033.stderr: 16800K .......... .......... .......... .......... .......... 22% 184M 0s 2019-12-09T09:43:03.959 INFO:teuthology.orchestra.run.smithi033.stderr: 16850K .......... .......... .......... .......... .......... 22% 193M 0s 2019-12-09T09:43:03.960 INFO:teuthology.orchestra.run.smithi033.stderr: 16900K .......... .......... .......... .......... .......... 23% 181M 0s 2019-12-09T09:43:03.960 INFO:teuthology.orchestra.run.smithi033.stderr: 16950K .......... .......... .......... .......... .......... 23% 159M 0s 2019-12-09T09:43:03.960 INFO:teuthology.orchestra.run.smithi033.stderr: 17000K .......... .......... .......... .......... .......... 23% 264M 0s 2019-12-09T09:43:03.960 INFO:teuthology.orchestra.run.smithi033.stderr: 17050K .......... .......... .......... .......... .......... 23% 193M 0s 2019-12-09T09:43:03.960 INFO:teuthology.orchestra.run.smithi033.stderr: 17100K .......... .......... .......... .......... .......... 23% 211M 0s 2019-12-09T09:43:03.960 INFO:teuthology.orchestra.run.smithi033.stderr: 17150K .......... .......... .......... .......... .......... 23% 182M 0s 2019-12-09T09:43:03.960 INFO:teuthology.orchestra.run.smithi033.stderr: 17200K .......... .......... .......... .......... .......... 23% 234M 0s 2019-12-09T09:43:03.961 INFO:teuthology.orchestra.run.smithi033.stderr: 17250K .......... .......... .......... .......... .......... 23% 231M 0s 2019-12-09T09:43:03.961 INFO:teuthology.orchestra.run.smithi033.stderr: 17300K .......... .......... .......... .......... .......... 23% 254M 0s 2019-12-09T09:43:03.961 INFO:teuthology.orchestra.run.smithi033.stderr: 17350K .......... .......... .......... .......... .......... 23% 227M 0s 2019-12-09T09:43:03.961 INFO:teuthology.orchestra.run.smithi033.stderr: 17400K .......... .......... .......... .......... .......... 23% 262M 0s 2019-12-09T09:43:03.961 INFO:teuthology.orchestra.run.smithi033.stderr: 17450K .......... .......... .......... .......... .......... 23% 279M 0s 2019-12-09T09:43:03.962 INFO:teuthology.orchestra.run.smithi033.stderr: 17500K .......... .......... .......... .......... .......... 23% 294M 0s 2019-12-09T09:43:03.962 INFO:teuthology.orchestra.run.smithi033.stderr: 17550K .......... .......... .......... .......... .......... 23% 248M 0s 2019-12-09T09:43:03.962 INFO:teuthology.orchestra.run.smithi033.stderr: 17600K .......... .......... .......... .......... .......... 23% 311M 0s 2019-12-09T09:43:03.962 INFO:teuthology.orchestra.run.smithi033.stderr: 17650K .......... .......... .......... .......... .......... 24% 317M 0s 2019-12-09T09:43:03.962 INFO:teuthology.orchestra.run.smithi033.stderr: 17700K .......... .......... .......... .......... .......... 24% 217M 0s 2019-12-09T09:43:03.962 INFO:teuthology.orchestra.run.smithi033.stderr: 17750K .......... .......... .......... .......... .......... 24% 156M 0s 2019-12-09T09:43:03.963 INFO:teuthology.orchestra.run.smithi033.stderr: 17800K .......... .......... .......... .......... .......... 24% 232M 0s 2019-12-09T09:43:03.963 INFO:teuthology.orchestra.run.smithi033.stderr: 17850K .......... .......... .......... .......... .......... 24% 216M 0s 2019-12-09T09:43:03.963 INFO:teuthology.orchestra.run.smithi033.stderr: 17900K .......... .......... .......... .......... .......... 24% 218M 0s 2019-12-09T09:43:03.963 INFO:teuthology.orchestra.run.smithi033.stderr: 17950K .......... .......... .......... .......... .......... 24% 190M 0s 2019-12-09T09:43:03.963 INFO:teuthology.orchestra.run.smithi033.stderr: 18000K .......... .......... .......... .......... .......... 24% 220M 0s 2019-12-09T09:43:03.963 INFO:teuthology.orchestra.run.smithi033.stderr: 18050K .......... .......... .......... .......... .......... 24% 214M 0s 2019-12-09T09:43:03.963 INFO:teuthology.orchestra.run.smithi033.stderr: 18100K .......... .......... .......... .......... .......... 24% 196M 0s 2019-12-09T09:43:03.964 INFO:teuthology.orchestra.run.smithi033.stderr: 18150K .......... .......... .......... .......... .......... 24% 181M 0s 2019-12-09T09:43:03.964 INFO:teuthology.orchestra.run.smithi033.stderr: 18200K .......... .......... .......... .......... .......... 24% 208M 0s 2019-12-09T09:43:03.964 INFO:teuthology.orchestra.run.smithi033.stderr: 18250K .......... .......... .......... .......... .......... 24% 195M 0s 2019-12-09T09:43:03.964 INFO:teuthology.orchestra.run.smithi033.stderr: 18300K .......... .......... .......... .......... .......... 24% 185M 0s 2019-12-09T09:43:03.964 INFO:teuthology.orchestra.run.smithi033.stderr: 18350K .......... .......... .......... .......... .......... 25% 165M 0s 2019-12-09T09:43:03.964 INFO:teuthology.orchestra.run.smithi033.stderr: 18400K .......... .......... .......... .......... .......... 25% 208M 0s 2019-12-09T09:43:03.965 INFO:teuthology.orchestra.run.smithi033.stderr: 18450K .......... .......... .......... .......... .......... 25% 200M 0s 2019-12-09T09:43:03.965 INFO:teuthology.orchestra.run.smithi033.stderr: 18500K .......... .......... .......... .......... .......... 25% 202M 0s 2019-12-09T09:43:03.965 INFO:teuthology.orchestra.run.smithi033.stderr: 18550K .......... .......... .......... .......... .......... 25% 174M 0s 2019-12-09T09:43:03.965 INFO:teuthology.orchestra.run.smithi033.stderr: 18600K .......... .......... .......... .......... .......... 25% 189M 0s 2019-12-09T09:43:03.965 INFO:teuthology.orchestra.run.smithi033.stderr: 18650K .......... .......... .......... .......... .......... 25% 211M 0s 2019-12-09T09:43:03.965 INFO:teuthology.orchestra.run.smithi033.stderr: 18700K .......... .......... .......... .......... .......... 25% 210M 0s 2019-12-09T09:43:03.965 INFO:teuthology.orchestra.run.smithi033.stderr: 18750K .......... .......... .......... .......... .......... 25% 176M 0s 2019-12-09T09:43:03.966 INFO:teuthology.orchestra.run.smithi033.stderr: 18800K .......... .......... .......... .......... .......... 25% 225M 0s 2019-12-09T09:43:03.966 INFO:teuthology.orchestra.run.smithi033.stderr: 18850K .......... .......... .......... .......... .......... 25% 220M 0s 2019-12-09T09:43:03.966 INFO:teuthology.orchestra.run.smithi033.stderr: 18900K .......... .......... .......... .......... .......... 25% 236M 0s 2019-12-09T09:43:03.966 INFO:teuthology.orchestra.run.smithi033.stderr: 18950K .......... .......... .......... .......... .......... 25% 177M 0s 2019-12-09T09:43:03.966 INFO:teuthology.orchestra.run.smithi033.stderr: 19000K .......... .......... .......... .......... .......... 25% 231M 0s 2019-12-09T09:43:03.966 INFO:teuthology.orchestra.run.smithi033.stderr: 19050K .......... .......... .......... .......... .......... 25% 220M 0s 2019-12-09T09:43:03.966 INFO:teuthology.orchestra.run.smithi033.stderr: 19100K .......... .......... .......... .......... .......... 26% 219M 0s 2019-12-09T09:43:03.967 INFO:teuthology.orchestra.run.smithi033.stderr: 19150K .......... .......... .......... .......... .......... 26% 147M 0s 2019-12-09T09:43:03.967 INFO:teuthology.orchestra.run.smithi033.stderr: 19200K .......... .......... .......... .......... .......... 26% 207M 0s 2019-12-09T09:43:03.967 INFO:teuthology.orchestra.run.smithi033.stderr: 19250K .......... .......... .......... .......... .......... 26% 202M 0s 2019-12-09T09:43:03.967 INFO:teuthology.orchestra.run.smithi033.stderr: 19300K .......... .......... .......... .......... .......... 26% 222M 0s 2019-12-09T09:43:03.967 INFO:teuthology.orchestra.run.smithi033.stderr: 19350K .......... .......... .......... .......... .......... 26% 184M 0s 2019-12-09T09:43:03.967 INFO:teuthology.orchestra.run.smithi033.stderr: 19400K .......... .......... .......... .......... .......... 26% 217M 0s 2019-12-09T09:43:03.967 INFO:teuthology.orchestra.run.smithi033.stderr: 19450K .......... .......... .......... .......... .......... 26% 218M 0s 2019-12-09T09:43:03.968 INFO:teuthology.orchestra.run.smithi033.stderr: 19500K .......... .......... .......... .......... .......... 26% 223M 0s 2019-12-09T09:43:03.968 INFO:teuthology.orchestra.run.smithi033.stderr: 19550K .......... .......... .......... .......... .......... 26% 177M 0s 2019-12-09T09:43:03.968 INFO:teuthology.orchestra.run.smithi033.stderr: 19600K .......... .......... .......... .......... .......... 26% 218M 0s 2019-12-09T09:43:03.968 INFO:teuthology.orchestra.run.smithi033.stderr: 19650K .......... .......... .......... .......... .......... 26% 204M 0s 2019-12-09T09:43:03.968 INFO:teuthology.orchestra.run.smithi033.stderr: 19700K .......... .......... .......... .......... .......... 26% 197M 0s 2019-12-09T09:43:03.968 INFO:teuthology.orchestra.run.smithi033.stderr: 19750K .......... .......... .......... .......... .......... 26% 216M 0s 2019-12-09T09:43:03.968 INFO:teuthology.orchestra.run.smithi033.stderr: 19800K .......... .......... .......... .......... .......... 26% 192M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 19850K .......... .......... .......... .......... .......... 27% 204M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 19900K .......... .......... .......... .......... .......... 27% 181M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 19950K .......... .......... .......... .......... .......... 27% 159M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 20000K .......... .......... .......... .......... .......... 27% 177M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 20050K .......... .......... .......... .......... .......... 27% 171M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 20100K .......... .......... .......... .......... .......... 27% 198M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 20150K .......... .......... .......... .......... .......... 27% 159M 0s 2019-12-09T09:43:03.969 INFO:teuthology.orchestra.run.smithi033.stderr: 20200K .......... .......... .......... .......... .......... 27% 177M 0s 2019-12-09T09:43:03.970 INFO:teuthology.orchestra.run.smithi033.stderr: 20250K .......... .......... .......... .......... .......... 27% 179M 0s 2019-12-09T09:43:03.970 INFO:teuthology.orchestra.run.smithi033.stderr: 20300K .......... .......... .......... .......... .......... 27% 200M 0s 2019-12-09T09:43:03.970 INFO:teuthology.orchestra.run.smithi033.stderr: 20350K .......... .......... .......... .......... .......... 27% 167M 0s 2019-12-09T09:43:03.970 INFO:teuthology.orchestra.run.smithi033.stderr: 20400K .......... .......... .......... .......... .......... 27% 180M 0s 2019-12-09T09:43:03.970 INFO:teuthology.orchestra.run.smithi033.stderr: 20450K .......... .......... .......... .......... .......... 27% 202M 0s 2019-12-09T09:43:03.970 INFO:teuthology.orchestra.run.smithi033.stderr: 20500K .......... .......... .......... .......... .......... 27% 206M 0s 2019-12-09T09:43:03.971 INFO:teuthology.orchestra.run.smithi033.stderr: 20550K .......... .......... .......... .......... .......... 28% 161M 0s 2019-12-09T09:43:03.971 INFO:teuthology.orchestra.run.smithi033.stderr: 20600K .......... .......... .......... .......... .......... 28% 193M 0s 2019-12-09T09:43:03.971 INFO:teuthology.orchestra.run.smithi033.stderr: 20650K .......... .......... .......... .......... .......... 28% 179M 0s 2019-12-09T09:43:03.971 INFO:teuthology.orchestra.run.smithi033.stderr: 20700K .......... .......... .......... .......... .......... 28% 170M 0s 2019-12-09T09:43:03.971 INFO:teuthology.orchestra.run.smithi033.stderr: 20750K .......... .......... .......... .......... .......... 28% 162M 0s 2019-12-09T09:43:03.972 INFO:teuthology.orchestra.run.smithi033.stderr: 20800K .......... .......... .......... .......... .......... 28% 192M 0s 2019-12-09T09:43:03.972 INFO:teuthology.orchestra.run.smithi033.stderr: 20850K .......... .......... .......... .......... .......... 28% 214M 0s 2019-12-09T09:43:03.972 INFO:teuthology.orchestra.run.smithi033.stderr: 20900K .......... .......... .......... .......... .......... 28% 184M 0s 2019-12-09T09:43:03.972 INFO:teuthology.orchestra.run.smithi033.stderr: 20950K .......... .......... .......... .......... .......... 28% 169M 0s 2019-12-09T09:43:03.972 INFO:teuthology.orchestra.run.smithi033.stderr: 21000K .......... .......... .......... .......... .......... 28% 214M 0s 2019-12-09T09:43:03.973 INFO:teuthology.orchestra.run.smithi033.stderr: 21050K .......... .......... .......... .......... .......... 28% 190M 0s 2019-12-09T09:43:03.973 INFO:teuthology.orchestra.run.smithi033.stderr: 21100K .......... .......... .......... .......... .......... 28% 174M 0s 2019-12-09T09:43:03.973 INFO:teuthology.orchestra.run.smithi033.stderr: 21150K .......... .......... .......... .......... .......... 28% 147M 0s 2019-12-09T09:43:03.973 INFO:teuthology.orchestra.run.smithi033.stderr: 21200K .......... .......... .......... .......... .......... 28% 195M 0s 2019-12-09T09:43:03.973 INFO:teuthology.orchestra.run.smithi033.stderr: 21250K .......... .......... .......... .......... .......... 28% 185M 0s 2019-12-09T09:43:03.973 INFO:teuthology.orchestra.run.smithi033.stderr: 21300K .......... .......... .......... .......... .......... 29% 155M 0s 2019-12-09T09:43:03.973 INFO:teuthology.orchestra.run.smithi033.stderr: 21350K .......... .......... .......... .......... .......... 29% 184M 0s 2019-12-09T09:43:03.974 INFO:teuthology.orchestra.run.smithi033.stderr: 21400K .......... .......... .......... .......... .......... 29% 147M 0s 2019-12-09T09:43:03.974 INFO:teuthology.orchestra.run.smithi033.stderr: 21450K .......... .......... .......... .......... .......... 29% 172M 0s 2019-12-09T09:43:03.974 INFO:teuthology.orchestra.run.smithi033.stderr: 21500K .......... .......... .......... .......... .......... 29% 189M 0s 2019-12-09T09:43:03.974 INFO:teuthology.orchestra.run.smithi033.stderr: 21550K .......... .......... .......... .......... .......... 29% 167M 0s 2019-12-09T09:43:03.974 INFO:teuthology.orchestra.run.smithi033.stderr: 21600K .......... .......... .......... .......... .......... 29% 187M 0s 2019-12-09T09:43:03.975 INFO:teuthology.orchestra.run.smithi033.stderr: 21650K .......... .......... .......... .......... .......... 29% 203M 0s 2019-12-09T09:43:03.975 INFO:teuthology.orchestra.run.smithi033.stderr: 21700K .......... .......... .......... .......... .......... 29% 284M 0s 2019-12-09T09:43:03.975 INFO:teuthology.orchestra.run.smithi033.stderr: 21750K .......... .......... .......... .......... .......... 29% 333M 0s 2019-12-09T09:43:03.975 INFO:teuthology.orchestra.run.smithi033.stderr: 21800K .......... .......... .......... .......... .......... 29% 286M 0s 2019-12-09T09:43:03.975 INFO:teuthology.orchestra.run.smithi033.stderr: 21850K .......... .......... .......... .......... .......... 29% 323M 0s 2019-12-09T09:43:03.975 INFO:teuthology.orchestra.run.smithi033.stderr: 21900K .......... .......... .......... .......... .......... 29% 226M 0s 2019-12-09T09:43:03.975 INFO:teuthology.orchestra.run.smithi033.stderr: 21950K .......... .......... .......... .......... .......... 29% 232M 0s 2019-12-09T09:43:03.976 INFO:teuthology.orchestra.run.smithi033.stderr: 22000K .......... .......... .......... .......... .......... 29% 355M 0s 2019-12-09T09:43:03.976 INFO:teuthology.orchestra.run.smithi033.stderr: 22050K .......... .......... .......... .......... .......... 30% 368M 0s 2019-12-09T09:43:03.976 INFO:teuthology.orchestra.run.smithi033.stderr: 22100K .......... .......... .......... .......... .......... 30% 247M 0s 2019-12-09T09:43:03.976 INFO:teuthology.orchestra.run.smithi033.stderr: 22150K .......... .......... .......... .......... .......... 30% 232M 0s 2019-12-09T09:43:03.976 INFO:teuthology.orchestra.run.smithi033.stderr: 22200K .......... .......... .......... .......... .......... 30% 365M 0s 2019-12-09T09:43:03.976 INFO:teuthology.orchestra.run.smithi033.stderr: 22250K .......... .......... .......... .......... .......... 30% 302M 0s 2019-12-09T09:43:03.976 INFO:teuthology.orchestra.run.smithi033.stderr: 22300K .......... .......... .......... .......... .......... 30% 325M 0s 2019-12-09T09:43:03.977 INFO:teuthology.orchestra.run.smithi033.stderr: 22350K .......... .......... .......... .......... .......... 30% 252M 0s 2019-12-09T09:43:03.977 INFO:teuthology.orchestra.run.smithi033.stderr: 22400K .......... .......... .......... .......... .......... 30% 297M 0s 2019-12-09T09:43:03.977 INFO:teuthology.orchestra.run.smithi033.stderr: 22450K .......... .......... .......... .......... .......... 30% 373M 0s 2019-12-09T09:43:03.977 INFO:teuthology.orchestra.run.smithi033.stderr: 22500K .......... .......... .......... .......... .......... 30% 269M 0s 2019-12-09T09:43:03.977 INFO:teuthology.orchestra.run.smithi033.stderr: 22550K .......... .......... .......... .......... .......... 30% 285M 0s 2019-12-09T09:43:03.977 INFO:teuthology.orchestra.run.smithi033.stderr: 22600K .......... .......... .......... .......... .......... 30% 207M 0s 2019-12-09T09:43:03.977 INFO:teuthology.orchestra.run.smithi033.stderr: 22650K .......... .......... .......... .......... .......... 30% 212M 0s 2019-12-09T09:43:03.978 INFO:teuthology.orchestra.run.smithi033.stderr: 22700K .......... .......... .......... .......... .......... 30% 273M 0s 2019-12-09T09:43:03.978 INFO:teuthology.orchestra.run.smithi033.stderr: 22750K .......... .......... .......... .......... .......... 30% 255M 0s 2019-12-09T09:43:03.978 INFO:teuthology.orchestra.run.smithi033.stderr: 22800K .......... .......... .......... .......... .......... 31% 338M 0s 2019-12-09T09:43:03.978 INFO:teuthology.orchestra.run.smithi033.stderr: 22850K .......... .......... .......... .......... .......... 31% 310M 0s 2019-12-09T09:43:03.978 INFO:teuthology.orchestra.run.smithi033.stderr: 22900K .......... .......... .......... .......... .......... 31% 339M 0s 2019-12-09T09:43:03.978 INFO:teuthology.orchestra.run.smithi033.stderr: 22950K .......... .......... .......... .......... .......... 31% 274M 0s 2019-12-09T09:43:03.978 INFO:teuthology.orchestra.run.smithi033.stderr: 23000K .......... .......... .......... .......... .......... 31% 287M 0s 2019-12-09T09:43:03.979 INFO:teuthology.orchestra.run.smithi033.stderr: 23050K .......... .......... .......... .......... .......... 31% 313M 0s 2019-12-09T09:43:03.979 INFO:teuthology.orchestra.run.smithi033.stderr: 23100K .......... .......... .......... .......... .......... 31% 338M 0s 2019-12-09T09:43:03.979 INFO:teuthology.orchestra.run.smithi033.stderr: 23150K .......... .......... .......... .......... .......... 31% 277M 0s 2019-12-09T09:43:03.979 INFO:teuthology.orchestra.run.smithi033.stderr: 23200K .......... .......... .......... .......... .......... 31% 344M 0s 2019-12-09T09:43:03.979 INFO:teuthology.orchestra.run.smithi033.stderr: 23250K .......... .......... .......... .......... .......... 31% 329M 0s 2019-12-09T09:43:03.979 INFO:teuthology.orchestra.run.smithi033.stderr: 23300K .......... .......... .......... .......... .......... 31% 301M 0s 2019-12-09T09:43:03.979 INFO:teuthology.orchestra.run.smithi033.stderr: 23350K .......... .......... .......... .......... .......... 31% 328M 0s 2019-12-09T09:43:03.980 INFO:teuthology.orchestra.run.smithi033.stderr: 23400K .......... .......... .......... .......... .......... 31% 315M 0s 2019-12-09T09:43:03.980 INFO:teuthology.orchestra.run.smithi033.stderr: 23450K .......... .......... .......... .......... .......... 31% 269M 0s 2019-12-09T09:43:03.980 INFO:teuthology.orchestra.run.smithi033.stderr: 23500K .......... .......... .......... .......... .......... 32% 239M 0s 2019-12-09T09:43:03.980 INFO:teuthology.orchestra.run.smithi033.stderr: 23550K .......... .......... .......... .......... .......... 32% 105M 0s 2019-12-09T09:43:03.980 INFO:teuthology.orchestra.run.smithi033.stderr: 23600K .......... .......... .......... .......... .......... 32% 211M 0s 2019-12-09T09:43:03.980 INFO:teuthology.orchestra.run.smithi033.stderr: 23650K .......... .......... .......... .......... .......... 32% 199M 0s 2019-12-09T09:43:03.980 INFO:teuthology.orchestra.run.smithi033.stderr: 23700K .......... .......... .......... .......... .......... 32% 198M 0s 2019-12-09T09:43:03.981 INFO:teuthology.orchestra.run.smithi033.stderr: 23750K .......... .......... .......... .......... .......... 32% 145M 0s 2019-12-09T09:43:03.981 INFO:teuthology.orchestra.run.smithi033.stderr: 23800K .......... .......... .......... .......... .......... 32% 167M 0s 2019-12-09T09:43:03.981 INFO:teuthology.orchestra.run.smithi033.stderr: 23850K .......... .......... .......... .......... .......... 32% 201M 0s 2019-12-09T09:43:03.981 INFO:teuthology.orchestra.run.smithi033.stderr: 23900K .......... .......... .......... .......... .......... 32% 196M 0s 2019-12-09T09:43:03.981 INFO:teuthology.orchestra.run.smithi033.stderr: 23950K .......... .......... .......... .......... .......... 32% 179M 0s 2019-12-09T09:43:03.981 INFO:teuthology.orchestra.run.smithi033.stderr: 24000K .......... .......... .......... .......... .......... 32% 203M 0s 2019-12-09T09:43:03.981 INFO:teuthology.orchestra.run.smithi033.stderr: 24050K .......... .......... .......... .......... .......... 32% 206M 0s 2019-12-09T09:43:03.982 INFO:teuthology.orchestra.run.smithi033.stderr: 24100K .......... .......... .......... .......... .......... 32% 203M 0s 2019-12-09T09:43:03.982 INFO:teuthology.orchestra.run.smithi033.stderr: 24150K .......... .......... .......... .......... .......... 32% 187M 0s 2019-12-09T09:43:03.982 INFO:teuthology.orchestra.run.smithi033.stderr: 24200K .......... .......... .......... .......... .......... 32% 197M 0s 2019-12-09T09:43:03.982 INFO:teuthology.orchestra.run.smithi033.stderr: 24250K .......... .......... .......... .......... .......... 33% 215M 0s 2019-12-09T09:43:03.982 INFO:teuthology.orchestra.run.smithi033.stderr: 24300K .......... .......... .......... .......... .......... 33% 231M 0s 2019-12-09T09:43:03.982 INFO:teuthology.orchestra.run.smithi033.stderr: 24350K .......... .......... .......... .......... .......... 33% 178M 0s 2019-12-09T09:43:03.982 INFO:teuthology.orchestra.run.smithi033.stderr: 24400K .......... .......... .......... .......... .......... 33% 210M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24450K .......... .......... .......... .......... .......... 33% 191M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24500K .......... .......... .......... .......... .......... 33% 196M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24550K .......... .......... .......... .......... .......... 33% 173M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24600K .......... .......... .......... .......... .......... 33% 193M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24650K .......... .......... .......... .......... .......... 33% 218M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24700K .......... .......... .......... .......... .......... 33% 199M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24750K .......... .......... .......... .......... .......... 33% 171M 0s 2019-12-09T09:43:03.983 INFO:teuthology.orchestra.run.smithi033.stderr: 24800K .......... .......... .......... .......... .......... 33% 204M 0s 2019-12-09T09:43:03.984 INFO:teuthology.orchestra.run.smithi033.stderr: 24850K .......... .......... .......... .......... .......... 33% 199M 0s 2019-12-09T09:43:03.984 INFO:teuthology.orchestra.run.smithi033.stderr: 24900K .......... .......... .......... .......... .......... 33% 214M 0s 2019-12-09T09:43:03.984 INFO:teuthology.orchestra.run.smithi033.stderr: 24950K .......... .......... .......... .......... .......... 33% 202M 0s 2019-12-09T09:43:03.984 INFO:teuthology.orchestra.run.smithi033.stderr: 25000K .......... .......... .......... .......... .......... 34% 218M 0s 2019-12-09T09:43:03.984 INFO:teuthology.orchestra.run.smithi033.stderr: 25050K .......... .......... .......... .......... .......... 34% 238M 0s 2019-12-09T09:43:03.984 INFO:teuthology.orchestra.run.smithi033.stderr: 25100K .......... .......... .......... .......... .......... 34% 233M 0s 2019-12-09T09:43:03.984 INFO:teuthology.orchestra.run.smithi033.stderr: 25150K .......... .......... .......... .......... .......... 34% 179M 0s 2019-12-09T09:43:03.985 INFO:teuthology.orchestra.run.smithi033.stderr: 25200K .......... .......... .......... .......... .......... 34% 208M 0s 2019-12-09T09:43:03.985 INFO:teuthology.orchestra.run.smithi033.stderr: 25250K .......... .......... .......... .......... .......... 34% 228M 0s 2019-12-09T09:43:03.985 INFO:teuthology.orchestra.run.smithi033.stderr: 25300K .......... .......... .......... .......... .......... 34% 228M 0s 2019-12-09T09:43:03.985 INFO:teuthology.orchestra.run.smithi033.stderr: 25350K .......... .......... .......... .......... .......... 34% 180M 0s 2019-12-09T09:43:03.985 INFO:teuthology.orchestra.run.smithi033.stderr: 25400K .......... .......... .......... .......... .......... 34% 212M 0s 2019-12-09T09:43:03.985 INFO:teuthology.orchestra.run.smithi033.stderr: 25450K .......... .......... .......... .......... .......... 34% 201M 0s 2019-12-09T09:43:03.985 INFO:teuthology.orchestra.run.smithi033.stderr: 25500K .......... .......... .......... .......... .......... 34% 235M 0s 2019-12-09T09:43:03.986 INFO:teuthology.orchestra.run.smithi033.stderr: 25550K .......... .......... .......... .......... .......... 34% 155M 0s 2019-12-09T09:43:03.986 INFO:teuthology.orchestra.run.smithi033.stderr: 25600K .......... .......... .......... .......... .......... 34% 174M 0s 2019-12-09T09:43:03.986 INFO:teuthology.orchestra.run.smithi033.stderr: 25650K .......... .......... .......... .......... .......... 34% 237M 0s 2019-12-09T09:43:03.986 INFO:teuthology.orchestra.run.smithi033.stderr: 25700K .......... .......... .......... .......... .......... 35% 188M 0s 2019-12-09T09:43:03.986 INFO:teuthology.orchestra.run.smithi033.stderr: 25750K .......... .......... .......... .......... .......... 35% 145M 0s 2019-12-09T09:43:03.986 INFO:teuthology.orchestra.run.smithi033.stderr: 25800K .......... .......... .......... .......... .......... 35% 225M 0s 2019-12-09T09:43:03.986 INFO:teuthology.orchestra.run.smithi033.stderr: 25850K .......... .......... .......... .......... .......... 35% 239M 0s 2019-12-09T09:43:03.987 INFO:teuthology.orchestra.run.smithi033.stderr: 25900K .......... .......... .......... .......... .......... 35% 187M 0s 2019-12-09T09:43:03.987 INFO:teuthology.orchestra.run.smithi033.stderr: 25950K .......... .......... .......... .......... .......... 35% 138M 0s 2019-12-09T09:43:03.987 INFO:teuthology.orchestra.run.smithi033.stderr: 26000K .......... .......... .......... .......... .......... 35% 193M 0s 2019-12-09T09:43:03.987 INFO:teuthology.orchestra.run.smithi033.stderr: 26050K .......... .......... .......... .......... .......... 35% 204M 0s 2019-12-09T09:43:03.987 INFO:teuthology.orchestra.run.smithi033.stderr: 26100K .......... .......... .......... .......... .......... 35% 170M 0s 2019-12-09T09:43:03.987 INFO:teuthology.orchestra.run.smithi033.stderr: 26150K .......... .......... .......... .......... .......... 35% 195M 0s 2019-12-09T09:43:03.987 INFO:teuthology.orchestra.run.smithi033.stderr: 26200K .......... .......... .......... .......... .......... 35% 248M 0s 2019-12-09T09:43:03.988 INFO:teuthology.orchestra.run.smithi033.stderr: 26250K .......... .......... .......... .......... .......... 35% 217M 0s 2019-12-09T09:43:03.988 INFO:teuthology.orchestra.run.smithi033.stderr: 26300K .......... .......... .......... .......... .......... 35% 198M 0s 2019-12-09T09:43:03.988 INFO:teuthology.orchestra.run.smithi033.stderr: 26350K .......... .......... .......... .......... .......... 35% 196M 0s 2019-12-09T09:43:03.988 INFO:teuthology.orchestra.run.smithi033.stderr: 26400K .......... .......... .......... .......... .......... 35% 211M 0s 2019-12-09T09:43:03.988 INFO:teuthology.orchestra.run.smithi033.stderr: 26450K .......... .......... .......... .......... .......... 36% 213M 0s 2019-12-09T09:43:03.988 INFO:teuthology.orchestra.run.smithi033.stderr: 26500K .......... .......... .......... .......... .......... 36% 207M 0s 2019-12-09T09:43:03.988 INFO:teuthology.orchestra.run.smithi033.stderr: 26550K .......... .......... .......... .......... .......... 36% 193M 0s 2019-12-09T09:43:03.989 INFO:teuthology.orchestra.run.smithi033.stderr: 26600K .......... .......... .......... .......... .......... 36% 203M 0s 2019-12-09T09:43:03.989 INFO:teuthology.orchestra.run.smithi033.stderr: 26650K .......... .......... .......... .......... .......... 36% 233M 0s 2019-12-09T09:43:03.989 INFO:teuthology.orchestra.run.smithi033.stderr: 26700K .......... .......... .......... .......... .......... 36% 225M 0s 2019-12-09T09:43:03.989 INFO:teuthology.orchestra.run.smithi033.stderr: 26750K .......... .......... .......... .......... .......... 36% 190M 0s 2019-12-09T09:43:03.989 INFO:teuthology.orchestra.run.smithi033.stderr: 26800K .......... .......... .......... .......... .......... 36% 241M 0s 2019-12-09T09:43:03.989 INFO:teuthology.orchestra.run.smithi033.stderr: 26850K .......... .......... .......... .......... .......... 36% 252M 0s 2019-12-09T09:43:03.989 INFO:teuthology.orchestra.run.smithi033.stderr: 26900K .......... .......... .......... .......... .......... 36% 242M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 26950K .......... .......... .......... .......... .......... 36% 204M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 27000K .......... .......... .......... .......... .......... 36% 241M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 27050K .......... .......... .......... .......... .......... 36% 245M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 27100K .......... .......... .......... .......... .......... 36% 263M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 27150K .......... .......... .......... .......... .......... 36% 212M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 27200K .......... .......... .......... .......... .......... 37% 239M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 27250K .......... .......... .......... .......... .......... 37% 256M 0s 2019-12-09T09:43:03.990 INFO:teuthology.orchestra.run.smithi033.stderr: 27300K .......... .......... .......... .......... .......... 37% 239M 0s 2019-12-09T09:43:03.991 INFO:teuthology.orchestra.run.smithi033.stderr: 27350K .......... .......... .......... .......... .......... 37% 209M 0s 2019-12-09T09:43:03.991 INFO:teuthology.orchestra.run.smithi033.stderr: 27400K .......... .......... .......... .......... .......... 37% 256M 0s 2019-12-09T09:43:03.991 INFO:teuthology.orchestra.run.smithi033.stderr: 27450K .......... .......... .......... .......... .......... 37% 240M 0s 2019-12-09T09:43:03.991 INFO:teuthology.orchestra.run.smithi033.stderr: 27500K .......... .......... .......... .......... .......... 37% 238M 0s 2019-12-09T09:43:03.991 INFO:teuthology.orchestra.run.smithi033.stderr: 27550K .......... .......... .......... .......... .......... 37% 190M 0s 2019-12-09T09:43:03.991 INFO:teuthology.orchestra.run.smithi033.stderr: 27600K .......... .......... .......... .......... .......... 37% 231M 0s 2019-12-09T09:43:03.991 INFO:teuthology.orchestra.run.smithi033.stderr: 27650K .......... .......... .......... .......... .......... 37% 227M 0s 2019-12-09T09:43:03.992 INFO:teuthology.orchestra.run.smithi033.stderr: 27700K .......... .......... .......... .......... .......... 37% 210M 0s 2019-12-09T09:43:03.992 INFO:teuthology.orchestra.run.smithi033.stderr: 27750K .......... .......... .......... .......... .......... 37% 211M 0s 2019-12-09T09:43:03.992 INFO:teuthology.orchestra.run.smithi033.stderr: 27800K .......... .......... .......... .......... .......... 37% 253M 0s 2019-12-09T09:43:03.992 INFO:teuthology.orchestra.run.smithi033.stderr: 27850K .......... .......... .......... .......... .......... 37% 239M 0s 2019-12-09T09:43:03.992 INFO:teuthology.orchestra.run.smithi033.stderr: 27900K .......... .......... .......... .......... .......... 37% 242M 0s 2019-12-09T09:43:03.992 INFO:teuthology.orchestra.run.smithi033.stderr: 27950K .......... .......... .......... .......... .......... 38% 217M 0s 2019-12-09T09:43:03.992 INFO:teuthology.orchestra.run.smithi033.stderr: 28000K .......... .......... .......... .......... .......... 38% 245M 0s 2019-12-09T09:43:03.993 INFO:teuthology.orchestra.run.smithi033.stderr: 28050K .......... .......... .......... .......... .......... 38% 236M 0s 2019-12-09T09:43:03.993 INFO:teuthology.orchestra.run.smithi033.stderr: 28100K .......... .......... .......... .......... .......... 38% 238M 0s 2019-12-09T09:43:03.993 INFO:teuthology.orchestra.run.smithi033.stderr: 28150K .......... .......... .......... .......... .......... 38% 227M 0s 2019-12-09T09:43:03.993 INFO:teuthology.orchestra.run.smithi033.stderr: 28200K .......... .......... .......... .......... .......... 38% 198M 0s 2019-12-09T09:43:03.993 INFO:teuthology.orchestra.run.smithi033.stderr: 28250K .......... .......... .......... .......... .......... 38% 199M 0s 2019-12-09T09:43:03.993 INFO:teuthology.orchestra.run.smithi033.stderr: 28300K .......... .......... .......... .......... .......... 38% 194M 0s 2019-12-09T09:43:03.993 INFO:teuthology.orchestra.run.smithi033.stderr: 28350K .......... .......... .......... .......... .......... 38% 201M 0s 2019-12-09T09:43:03.994 INFO:teuthology.orchestra.run.smithi033.stderr: 28400K .......... .......... .......... .......... .......... 38% 229M 0s 2019-12-09T09:43:03.994 INFO:teuthology.orchestra.run.smithi033.stderr: 28450K .......... .......... .......... .......... .......... 38% 227M 0s 2019-12-09T09:43:03.994 INFO:teuthology.orchestra.run.smithi033.stderr: 28500K .......... .......... .......... .......... .......... 38% 202M 0s 2019-12-09T09:43:03.994 INFO:teuthology.orchestra.run.smithi033.stderr: 28550K .......... .......... .......... .......... .......... 38% 174M 0s 2019-12-09T09:43:03.994 INFO:teuthology.orchestra.run.smithi033.stderr: 28600K .......... .......... .......... .......... .......... 38% 226M 0s 2019-12-09T09:43:03.994 INFO:teuthology.orchestra.run.smithi033.stderr: 28650K .......... .......... .......... .......... .......... 39% 238M 0s 2019-12-09T09:43:03.995 INFO:teuthology.orchestra.run.smithi033.stderr: 28700K .......... .......... .......... .......... .......... 39% 221M 0s 2019-12-09T09:43:03.995 INFO:teuthology.orchestra.run.smithi033.stderr: 28750K .......... .......... .......... .......... .......... 39% 182M 0s 2019-12-09T09:43:03.995 INFO:teuthology.orchestra.run.smithi033.stderr: 28800K .......... .......... .......... .......... .......... 39% 219M 0s 2019-12-09T09:43:03.995 INFO:teuthology.orchestra.run.smithi033.stderr: 28850K .......... .......... .......... .......... .......... 39% 228M 0s 2019-12-09T09:43:03.995 INFO:teuthology.orchestra.run.smithi033.stderr: 28900K .......... .......... .......... .......... .......... 39% 215M 0s 2019-12-09T09:43:03.995 INFO:teuthology.orchestra.run.smithi033.stderr: 28950K .......... .......... .......... .......... .......... 39% 203M 0s 2019-12-09T09:43:03.996 INFO:teuthology.orchestra.run.smithi033.stderr: 29000K .......... .......... .......... .......... .......... 39% 219M 0s 2019-12-09T09:43:03.996 INFO:teuthology.orchestra.run.smithi033.stderr: 29050K .......... .......... .......... .......... .......... 39% 229M 0s 2019-12-09T09:43:03.996 INFO:teuthology.orchestra.run.smithi033.stderr: 29100K .......... .......... .......... .......... .......... 39% 239M 0s 2019-12-09T09:43:03.996 INFO:teuthology.orchestra.run.smithi033.stderr: 29150K .......... .......... .......... .......... .......... 39% 208M 0s 2019-12-09T09:43:03.996 INFO:teuthology.orchestra.run.smithi033.stderr: 29200K .......... .......... .......... .......... .......... 39% 245M 0s 2019-12-09T09:43:03.996 INFO:teuthology.orchestra.run.smithi033.stderr: 29250K .......... .......... .......... .......... .......... 39% 245M 0s 2019-12-09T09:43:03.997 INFO:teuthology.orchestra.run.smithi033.stderr: 29300K .......... .......... .......... .......... .......... 39% 212M 0s 2019-12-09T09:43:03.997 INFO:teuthology.orchestra.run.smithi033.stderr: 29350K .......... .......... .......... .......... .......... 39% 200M 0s 2019-12-09T09:43:03.997 INFO:teuthology.orchestra.run.smithi033.stderr: 29400K .......... .......... .......... .......... .......... 40% 240M 0s 2019-12-09T09:43:03.997 INFO:teuthology.orchestra.run.smithi033.stderr: 29450K .......... .......... .......... .......... .......... 40% 214M 0s 2019-12-09T09:43:03.997 INFO:teuthology.orchestra.run.smithi033.stderr: 29500K .......... .......... .......... .......... .......... 40% 208M 0s 2019-12-09T09:43:03.997 INFO:teuthology.orchestra.run.smithi033.stderr: 29550K .......... .......... .......... .......... .......... 40% 197M 0s 2019-12-09T09:43:03.998 INFO:teuthology.orchestra.run.smithi033.stderr: 29600K .......... .......... .......... .......... .......... 40% 244M 0s 2019-12-09T09:43:03.998 INFO:teuthology.orchestra.run.smithi033.stderr: 29650K .......... .......... .......... .......... .......... 40% 309M 0s 2019-12-09T09:43:03.998 INFO:teuthology.orchestra.run.smithi033.stderr: 29700K .......... .......... .......... .......... .......... 40% 347M 0s 2019-12-09T09:43:03.998 INFO:teuthology.orchestra.run.smithi033.stderr: 29750K .......... .......... .......... .......... .......... 40% 357M 0s 2019-12-09T09:43:03.998 INFO:teuthology.orchestra.run.smithi033.stderr: 29800K .......... .......... .......... .......... .......... 40% 400M 0s 2019-12-09T09:43:03.998 INFO:teuthology.orchestra.run.smithi033.stderr: 29850K .......... .......... .......... .......... .......... 40% 311M 0s 2019-12-09T09:43:03.998 INFO:teuthology.orchestra.run.smithi033.stderr: 29900K .......... .......... .......... .......... .......... 40% 335M 0s 2019-12-09T09:43:03.999 INFO:teuthology.orchestra.run.smithi033.stderr: 29950K .......... .......... .......... .......... .......... 40% 276M 0s 2019-12-09T09:43:03.999 INFO:teuthology.orchestra.run.smithi033.stderr: 30000K .......... .......... .......... .......... .......... 40% 262M 0s 2019-12-09T09:43:03.999 INFO:teuthology.orchestra.run.smithi033.stderr: 30050K .......... .......... .......... .......... .......... 40% 297M 0s 2019-12-09T09:43:03.999 INFO:teuthology.orchestra.run.smithi033.stderr: 30100K .......... .......... .......... .......... .......... 40% 302M 0s 2019-12-09T09:43:03.999 INFO:teuthology.orchestra.run.smithi033.stderr: 30150K .......... .......... .......... .......... .......... 41% 245M 0s 2019-12-09T09:43:03.999 INFO:teuthology.orchestra.run.smithi033.stderr: 30200K .......... .......... .......... .......... .......... 41% 382M 0s 2019-12-09T09:43:03.999 INFO:teuthology.orchestra.run.smithi033.stderr: 30250K .......... .......... .......... .......... .......... 41% 431M 0s 2019-12-09T09:43:04.000 INFO:teuthology.orchestra.run.smithi033.stderr: 30300K .......... .......... .......... .......... .......... 41% 410M 0s 2019-12-09T09:43:04.000 INFO:teuthology.orchestra.run.smithi033.stderr: 30350K .......... .......... .......... .......... .......... 41% 276M 0s 2019-12-09T09:43:04.000 INFO:teuthology.orchestra.run.smithi033.stderr: 30400K .......... .......... .......... .......... .......... 41% 353M 0s 2019-12-09T09:43:04.000 INFO:teuthology.orchestra.run.smithi033.stderr: 30450K .......... .......... .......... .......... .......... 41% 375M 0s 2019-12-09T09:43:04.000 INFO:teuthology.orchestra.run.smithi033.stderr: 30500K .......... .......... .......... .......... .......... 41% 328M 0s 2019-12-09T09:43:04.001 INFO:teuthology.orchestra.run.smithi033.stderr: 30550K .......... .......... .......... .......... .......... 41% 251M 0s 2019-12-09T09:43:04.001 INFO:teuthology.orchestra.run.smithi033.stderr: 30600K .......... .......... .......... .......... .......... 41% 396M 0s 2019-12-09T09:43:04.001 INFO:teuthology.orchestra.run.smithi033.stderr: 30650K .......... .......... .......... .......... .......... 41% 302M 0s 2019-12-09T09:43:04.001 INFO:teuthology.orchestra.run.smithi033.stderr: 30700K .......... .......... .......... .......... .......... 41% 314M 0s 2019-12-09T09:43:04.001 INFO:teuthology.orchestra.run.smithi033.stderr: 30750K .......... .......... .......... .......... .......... 41% 282M 0s 2019-12-09T09:43:04.001 INFO:teuthology.orchestra.run.smithi033.stderr: 30800K .......... .......... .......... .......... .......... 41% 418M 0s 2019-12-09T09:43:04.001 INFO:teuthology.orchestra.run.smithi033.stderr: 30850K .......... .......... .......... .......... .......... 42% 302M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 30900K .......... .......... .......... .......... .......... 42% 326M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 30950K .......... .......... .......... .......... .......... 42% 330M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 31000K .......... .......... .......... .......... .......... 42% 326M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 31050K .......... .......... .......... .......... .......... 42% 366M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 31100K .......... .......... .......... .......... .......... 42% 334M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 31150K .......... .......... .......... .......... .......... 42% 281M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 31200K .......... .......... .......... .......... .......... 42% 303M 0s 2019-12-09T09:43:04.002 INFO:teuthology.orchestra.run.smithi033.stderr: 31250K .......... .......... .......... .......... .......... 42% 302M 0s 2019-12-09T09:43:04.003 INFO:teuthology.orchestra.run.smithi033.stderr: 31300K .......... .......... .......... .......... .......... 42% 335M 0s 2019-12-09T09:43:04.003 INFO:teuthology.orchestra.run.smithi033.stderr: 31350K .......... .......... .......... .......... .......... 42% 335M 0s 2019-12-09T09:43:04.003 INFO:teuthology.orchestra.run.smithi033.stderr: 31400K .......... .......... .......... .......... .......... 42% 362M 0s 2019-12-09T09:43:04.003 INFO:teuthology.orchestra.run.smithi033.stderr: 31450K .......... .......... .......... .......... .......... 42% 391M 0s 2019-12-09T09:43:04.003 INFO:teuthology.orchestra.run.smithi033.stderr: 31500K .......... .......... .......... .......... .......... 42% 395M 0s 2019-12-09T09:43:04.003 INFO:teuthology.orchestra.run.smithi033.stderr: 31550K .......... .......... .......... .......... .......... 42% 331M 0s 2019-12-09T09:43:04.003 INFO:teuthology.orchestra.run.smithi033.stderr: 31600K .......... .......... .......... .......... .......... 43% 329M 0s 2019-12-09T09:43:04.004 INFO:teuthology.orchestra.run.smithi033.stderr: 31650K .......... .......... .......... .......... .......... 43% 339M 0s 2019-12-09T09:43:04.004 INFO:teuthology.orchestra.run.smithi033.stderr: 31700K .......... .......... .......... .......... .......... 43% 337M 0s 2019-12-09T09:43:04.004 INFO:teuthology.orchestra.run.smithi033.stderr: 31750K .......... .......... .......... .......... .......... 43% 252M 0s 2019-12-09T09:43:04.004 INFO:teuthology.orchestra.run.smithi033.stderr: 31800K .......... .......... .......... .......... .......... 43% 402M 0s 2019-12-09T09:43:04.004 INFO:teuthology.orchestra.run.smithi033.stderr: 31850K .......... .......... .......... .......... .......... 43% 314M 0s 2019-12-09T09:43:04.004 INFO:teuthology.orchestra.run.smithi033.stderr: 31900K .......... .......... .......... .......... .......... 43% 307M 0s 2019-12-09T09:43:04.004 INFO:teuthology.orchestra.run.smithi033.stderr: 31950K .......... .......... .......... .......... .......... 43% 345M 0s 2019-12-09T09:43:04.005 INFO:teuthology.orchestra.run.smithi033.stderr: 32000K .......... .......... .......... .......... .......... 43% 318M 0s 2019-12-09T09:43:04.005 INFO:teuthology.orchestra.run.smithi033.stderr: 32050K .......... .......... .......... .......... .......... 43% 306M 0s 2019-12-09T09:43:04.005 INFO:teuthology.orchestra.run.smithi033.stderr: 32100K .......... .......... .......... .......... .......... 43% 318M 0s 2019-12-09T09:43:04.005 INFO:teuthology.orchestra.run.smithi033.stderr: 32150K .......... .......... .......... .......... .......... 43% 306M 0s 2019-12-09T09:43:04.005 INFO:teuthology.orchestra.run.smithi033.stderr: 32200K .......... .......... .......... .......... .......... 43% 363M 0s 2019-12-09T09:43:04.005 INFO:teuthology.orchestra.run.smithi033.stderr: 32250K .......... .......... .......... .......... .......... 43% 394M 0s 2019-12-09T09:43:04.005 INFO:teuthology.orchestra.run.smithi033.stderr: 32300K .......... .......... .......... .......... .......... 43% 296M 0s 2019-12-09T09:43:04.006 INFO:teuthology.orchestra.run.smithi033.stderr: 32350K .......... .......... .......... .......... .......... 44% 266M 0s 2019-12-09T09:43:04.006 INFO:teuthology.orchestra.run.smithi033.stderr: 32400K .......... .......... .......... .......... .......... 44% 284M 0s 2019-12-09T09:43:04.006 INFO:teuthology.orchestra.run.smithi033.stderr: 32450K .......... .......... .......... .......... .......... 44% 301M 0s 2019-12-09T09:43:04.006 INFO:teuthology.orchestra.run.smithi033.stderr: 32500K .......... .......... .......... .......... .......... 44% 284M 0s 2019-12-09T09:43:04.006 INFO:teuthology.orchestra.run.smithi033.stderr: 32550K .......... .......... .......... .......... .......... 44% 249M 0s 2019-12-09T09:43:04.006 INFO:teuthology.orchestra.run.smithi033.stderr: 32600K .......... .......... .......... .......... .......... 44% 347M 0s 2019-12-09T09:43:04.006 INFO:teuthology.orchestra.run.smithi033.stderr: 32650K .......... .......... .......... .......... .......... 44% 349M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 32700K .......... .......... .......... .......... .......... 44% 347M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 32750K .......... .......... .......... .......... .......... 44% 292M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 32800K .......... .......... .......... .......... .......... 44% 352M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 32850K .......... .......... .......... .......... .......... 44% 368M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 32900K .......... .......... .......... .......... .......... 44% 287M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 32950K .......... .......... .......... .......... .......... 44% 301M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 33000K .......... .......... .......... .......... .......... 44% 355M 0s 2019-12-09T09:43:04.007 INFO:teuthology.orchestra.run.smithi033.stderr: 33050K .......... .......... .......... .......... .......... 44% 319M 0s 2019-12-09T09:43:04.008 INFO:teuthology.orchestra.run.smithi033.stderr: 33100K .......... .......... .......... .......... .......... 45% 362M 0s 2019-12-09T09:43:04.008 INFO:teuthology.orchestra.run.smithi033.stderr: 33150K .......... .......... .......... .......... .......... 45% 205M 0s 2019-12-09T09:43:04.008 INFO:teuthology.orchestra.run.smithi033.stderr: 33200K .......... .......... .......... .......... .......... 45% 251M 0s 2019-12-09T09:43:04.008 INFO:teuthology.orchestra.run.smithi033.stderr: 33250K .......... .......... .......... .......... .......... 45% 255M 0s 2019-12-09T09:43:04.008 INFO:teuthology.orchestra.run.smithi033.stderr: 33300K .......... .......... .......... .......... .......... 45% 264M 0s 2019-12-09T09:43:04.008 INFO:teuthology.orchestra.run.smithi033.stderr: 33350K .......... .......... .......... .......... .......... 45% 208M 0s 2019-12-09T09:43:04.008 INFO:teuthology.orchestra.run.smithi033.stderr: 33400K .......... .......... .......... .......... .......... 45% 220M 0s 2019-12-09T09:43:04.009 INFO:teuthology.orchestra.run.smithi033.stderr: 33450K .......... .......... .......... .......... .......... 45% 262M 0s 2019-12-09T09:43:04.009 INFO:teuthology.orchestra.run.smithi033.stderr: 33500K .......... .......... .......... .......... .......... 45% 229M 0s 2019-12-09T09:43:04.009 INFO:teuthology.orchestra.run.smithi033.stderr: 33550K .......... .......... .......... .......... .......... 45% 189M 0s 2019-12-09T09:43:04.009 INFO:teuthology.orchestra.run.smithi033.stderr: 33600K .......... .......... .......... .......... .......... 45% 222M 0s 2019-12-09T09:43:04.009 INFO:teuthology.orchestra.run.smithi033.stderr: 33650K .......... .......... .......... .......... .......... 45% 210M 0s 2019-12-09T09:43:04.009 INFO:teuthology.orchestra.run.smithi033.stderr: 33700K .......... .......... .......... .......... .......... 45% 240M 0s 2019-12-09T09:43:04.010 INFO:teuthology.orchestra.run.smithi033.stderr: 33750K .......... .......... .......... .......... .......... 45% 195M 0s 2019-12-09T09:43:04.010 INFO:teuthology.orchestra.run.smithi033.stderr: 33800K .......... .......... .......... .......... .......... 46% 223M 0s 2019-12-09T09:43:04.010 INFO:teuthology.orchestra.run.smithi033.stderr: 33850K .......... .......... .......... .......... .......... 46% 229M 0s 2019-12-09T09:43:04.010 INFO:teuthology.orchestra.run.smithi033.stderr: 33900K .......... .......... .......... .......... .......... 46% 219M 0s 2019-12-09T09:43:04.010 INFO:teuthology.orchestra.run.smithi033.stderr: 33950K .......... .......... .......... .......... .......... 46% 203M 0s 2019-12-09T09:43:04.010 INFO:teuthology.orchestra.run.smithi033.stderr: 34000K .......... .......... .......... .......... .......... 46% 232M 0s 2019-12-09T09:43:04.011 INFO:teuthology.orchestra.run.smithi033.stderr: 34050K .......... .......... .......... .......... .......... 46% 226M 0s 2019-12-09T09:43:04.011 INFO:teuthology.orchestra.run.smithi033.stderr: 34100K .......... .......... .......... .......... .......... 46% 202M 0s 2019-12-09T09:43:04.011 INFO:teuthology.orchestra.run.smithi033.stderr: 34150K .......... .......... .......... .......... .......... 46% 183M 0s 2019-12-09T09:43:04.011 INFO:teuthology.orchestra.run.smithi033.stderr: 34200K .......... .......... .......... .......... .......... 46% 184M 0s 2019-12-09T09:43:04.011 INFO:teuthology.orchestra.run.smithi033.stderr: 34250K .......... .......... .......... .......... .......... 46% 155M 0s 2019-12-09T09:43:04.011 INFO:teuthology.orchestra.run.smithi033.stderr: 34300K .......... .......... .......... .......... .......... 46% 177M 0s 2019-12-09T09:43:04.011 INFO:teuthology.orchestra.run.smithi033.stderr: 34350K .......... .......... .......... .......... .......... 46% 142M 0s 2019-12-09T09:43:04.012 INFO:teuthology.orchestra.run.smithi033.stderr: 34400K .......... .......... .......... .......... .......... 46% 169M 0s 2019-12-09T09:43:04.012 INFO:teuthology.orchestra.run.smithi033.stderr: 34450K .......... .......... .......... .......... .......... 46% 156M 0s 2019-12-09T09:43:04.012 INFO:teuthology.orchestra.run.smithi033.stderr: 34500K .......... .......... .......... .......... .......... 46% 217M 0s 2019-12-09T09:43:04.012 INFO:teuthology.orchestra.run.smithi033.stderr: 34550K .......... .......... .......... .......... .......... 47% 198M 0s 2019-12-09T09:43:04.012 INFO:teuthology.orchestra.run.smithi033.stderr: 34600K .......... .......... .......... .......... .......... 47% 195M 0s 2019-12-09T09:43:04.012 INFO:teuthology.orchestra.run.smithi033.stderr: 34650K .......... .......... .......... .......... .......... 47% 209M 0s 2019-12-09T09:43:04.012 INFO:teuthology.orchestra.run.smithi033.stderr: 34700K .......... .......... .......... .......... .......... 47% 214M 0s 2019-12-09T09:43:04.013 INFO:teuthology.orchestra.run.smithi033.stderr: 34750K .......... .......... .......... .......... .......... 47% 191M 0s 2019-12-09T09:43:04.013 INFO:teuthology.orchestra.run.smithi033.stderr: 34800K .......... .......... .......... .......... .......... 47% 214M 0s 2019-12-09T09:43:04.013 INFO:teuthology.orchestra.run.smithi033.stderr: 34850K .......... .......... .......... .......... .......... 47% 220M 0s 2019-12-09T09:43:04.013 INFO:teuthology.orchestra.run.smithi033.stderr: 34900K .......... .......... .......... .......... .......... 47% 227M 0s 2019-12-09T09:43:04.013 INFO:teuthology.orchestra.run.smithi033.stderr: 34950K .......... .......... .......... .......... .......... 47% 186M 0s 2019-12-09T09:43:04.013 INFO:teuthology.orchestra.run.smithi033.stderr: 35000K .......... .......... .......... .......... .......... 47% 197M 0s 2019-12-09T09:43:04.013 INFO:teuthology.orchestra.run.smithi033.stderr: 35050K .......... .......... .......... .......... .......... 47% 231M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35100K .......... .......... .......... .......... .......... 47% 222M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35150K .......... .......... .......... .......... .......... 47% 182M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35200K .......... .......... .......... .......... .......... 47% 237M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35250K .......... .......... .......... .......... .......... 47% 232M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35300K .......... .......... .......... .......... .......... 48% 213M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35350K .......... .......... .......... .......... .......... 48% 198M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35400K .......... .......... .......... .......... .......... 48% 222M 0s 2019-12-09T09:43:04.014 INFO:teuthology.orchestra.run.smithi033.stderr: 35450K .......... .......... .......... .......... .......... 48% 209M 0s 2019-12-09T09:43:04.015 INFO:teuthology.orchestra.run.smithi033.stderr: 35500K .......... .......... .......... .......... .......... 48% 232M 0s 2019-12-09T09:43:04.015 INFO:teuthology.orchestra.run.smithi033.stderr: 35550K .......... .......... .......... .......... .......... 48% 184M 0s 2019-12-09T09:43:04.015 INFO:teuthology.orchestra.run.smithi033.stderr: 35600K .......... .......... .......... .......... .......... 48% 228M 0s 2019-12-09T09:43:04.015 INFO:teuthology.orchestra.run.smithi033.stderr: 35650K .......... .......... .......... .......... .......... 48% 216M 0s 2019-12-09T09:43:04.015 INFO:teuthology.orchestra.run.smithi033.stderr: 35700K .......... .......... .......... .......... .......... 48% 218M 0s 2019-12-09T09:43:04.015 INFO:teuthology.orchestra.run.smithi033.stderr: 35750K .......... .......... .......... .......... .......... 48% 198M 0s 2019-12-09T09:43:04.015 INFO:teuthology.orchestra.run.smithi033.stderr: 35800K .......... .......... .......... .......... .......... 48% 223M 0s 2019-12-09T09:43:04.016 INFO:teuthology.orchestra.run.smithi033.stderr: 35850K .......... .......... .......... .......... .......... 48% 226M 0s 2019-12-09T09:43:04.016 INFO:teuthology.orchestra.run.smithi033.stderr: 35900K .......... .......... .......... .......... .......... 48% 206M 0s 2019-12-09T09:43:04.016 INFO:teuthology.orchestra.run.smithi033.stderr: 35950K .......... .......... .......... .......... .......... 48% 205M 0s 2019-12-09T09:43:04.016 INFO:teuthology.orchestra.run.smithi033.stderr: 36000K .......... .......... .......... .......... .......... 49% 237M 0s 2019-12-09T09:43:04.016 INFO:teuthology.orchestra.run.smithi033.stderr: 36050K .......... .......... .......... .......... .......... 49% 212M 0s 2019-12-09T09:43:04.016 INFO:teuthology.orchestra.run.smithi033.stderr: 36100K .......... .......... .......... .......... .......... 49% 208M 0s 2019-12-09T09:43:04.016 INFO:teuthology.orchestra.run.smithi033.stderr: 36150K .......... .......... .......... .......... .......... 49% 301M 0s 2019-12-09T09:43:04.017 INFO:teuthology.orchestra.run.smithi033.stderr: 36200K .......... .......... .......... .......... .......... 49% 319M 0s 2019-12-09T09:43:04.017 INFO:teuthology.orchestra.run.smithi033.stderr: 36250K .......... .......... .......... .......... .......... 49% 337M 0s 2019-12-09T09:43:04.017 INFO:teuthology.orchestra.run.smithi033.stderr: 36300K .......... .......... .......... .......... .......... 49% 326M 0s 2019-12-09T09:43:04.017 INFO:teuthology.orchestra.run.smithi033.stderr: 36350K .......... .......... .......... .......... .......... 49% 299M 0s 2019-12-09T09:43:04.017 INFO:teuthology.orchestra.run.smithi033.stderr: 36400K .......... .......... .......... .......... .......... 49% 300M 0s 2019-12-09T09:43:04.017 INFO:teuthology.orchestra.run.smithi033.stderr: 36450K .......... .......... .......... .......... .......... 49% 339M 0s 2019-12-09T09:43:04.017 INFO:teuthology.orchestra.run.smithi033.stderr: 36500K .......... .......... .......... .......... .......... 49% 326M 0s 2019-12-09T09:43:04.018 INFO:teuthology.orchestra.run.smithi033.stderr: 36550K .......... .......... .......... .......... .......... 49% 287M 0s 2019-12-09T09:43:04.018 INFO:teuthology.orchestra.run.smithi033.stderr: 36600K .......... .......... .......... .......... .......... 49% 287M 0s 2019-12-09T09:43:04.018 INFO:teuthology.orchestra.run.smithi033.stderr: 36650K .......... .......... .......... .......... .......... 49% 335M 0s 2019-12-09T09:43:04.018 INFO:teuthology.orchestra.run.smithi033.stderr: 36700K .......... .......... .......... .......... .......... 49% 267M 0s 2019-12-09T09:43:04.018 INFO:teuthology.orchestra.run.smithi033.stderr: 36750K .......... .......... .......... .......... .......... 50% 246M 0s 2019-12-09T09:43:04.018 INFO:teuthology.orchestra.run.smithi033.stderr: 36800K .......... .......... .......... .......... .......... 50% 350M 0s 2019-12-09T09:43:04.018 INFO:teuthology.orchestra.run.smithi033.stderr: 36850K .......... .......... .......... .......... .......... 50% 301M 0s 2019-12-09T09:43:04.019 INFO:teuthology.orchestra.run.smithi033.stderr: 36900K .......... .......... .......... .......... .......... 50% 392M 0s 2019-12-09T09:43:04.019 INFO:teuthology.orchestra.run.smithi033.stderr: 36950K .......... .......... .......... .......... .......... 50% 348M 0s 2019-12-09T09:43:04.019 INFO:teuthology.orchestra.run.smithi033.stderr: 37000K .......... .......... .......... .......... .......... 50% 390M 0s 2019-12-09T09:43:04.019 INFO:teuthology.orchestra.run.smithi033.stderr: 37050K .......... .......... .......... .......... .......... 50% 341M 0s 2019-12-09T09:43:04.019 INFO:teuthology.orchestra.run.smithi033.stderr: 37100K .......... .......... .......... .......... .......... 50% 358M 0s 2019-12-09T09:43:04.019 INFO:teuthology.orchestra.run.smithi033.stderr: 37150K .......... .......... .......... .......... .......... 50% 345M 0s 2019-12-09T09:43:04.019 INFO:teuthology.orchestra.run.smithi033.stderr: 37200K .......... .......... .......... .......... .......... 50% 409M 0s 2019-12-09T09:43:04.020 INFO:teuthology.orchestra.run.smithi033.stderr: 37250K .......... .......... .......... .......... .......... 50% 385M 0s 2019-12-09T09:43:04.020 INFO:teuthology.orchestra.run.smithi033.stderr: 37300K .......... .......... .......... .......... .......... 50% 415M 0s 2019-12-09T09:43:04.020 INFO:teuthology.orchestra.run.smithi033.stderr: 37350K .......... .......... .......... .......... .......... 50% 367M 0s 2019-12-09T09:43:04.020 INFO:teuthology.orchestra.run.smithi033.stderr: 37400K .......... .......... .......... .......... .......... 50% 410M 0s 2019-12-09T09:43:04.020 INFO:teuthology.orchestra.run.smithi033.stderr: 37450K .......... .......... .......... .......... .......... 50% 328M 0s 2019-12-09T09:43:04.020 INFO:teuthology.orchestra.run.smithi033.stderr: 37500K .......... .......... .......... .......... .......... 51% 316M 0s 2019-12-09T09:43:04.020 INFO:teuthology.orchestra.run.smithi033.stderr: 37550K .......... .......... .......... .......... .......... 51% 298M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37600K .......... .......... .......... .......... .......... 51% 331M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37650K .......... .......... .......... .......... .......... 51% 328M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37700K .......... .......... .......... .......... .......... 51% 315M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37750K .......... .......... .......... .......... .......... 51% 320M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37800K .......... .......... .......... .......... .......... 51% 331M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37850K .......... .......... .......... .......... .......... 51% 317M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37900K .......... .......... .......... .......... .......... 51% 367M 0s 2019-12-09T09:43:04.021 INFO:teuthology.orchestra.run.smithi033.stderr: 37950K .......... .......... .......... .......... .......... 51% 287M 0s 2019-12-09T09:43:04.022 INFO:teuthology.orchestra.run.smithi033.stderr: 38000K .......... .......... .......... .......... .......... 51% 320M 0s 2019-12-09T09:43:04.022 INFO:teuthology.orchestra.run.smithi033.stderr: 38050K .......... .......... .......... .......... .......... 51% 332M 0s 2019-12-09T09:43:04.022 INFO:teuthology.orchestra.run.smithi033.stderr: 38100K .......... .......... .......... .......... .......... 51% 327M 0s 2019-12-09T09:43:04.022 INFO:teuthology.orchestra.run.smithi033.stderr: 38150K .......... .......... .......... .......... .......... 51% 230M 0s 2019-12-09T09:43:04.022 INFO:teuthology.orchestra.run.smithi033.stderr: 38200K .......... .......... .......... .......... .......... 51% 405M 0s 2019-12-09T09:43:04.022 INFO:teuthology.orchestra.run.smithi033.stderr: 38250K .......... .......... .......... .......... .......... 52% 305M 0s 2019-12-09T09:43:04.022 INFO:teuthology.orchestra.run.smithi033.stderr: 38300K .......... .......... .......... .......... .......... 52% 371M 0s 2019-12-09T09:43:04.023 INFO:teuthology.orchestra.run.smithi033.stderr: 38350K .......... .......... .......... .......... .......... 52% 256M 0s 2019-12-09T09:43:04.023 INFO:teuthology.orchestra.run.smithi033.stderr: 38400K .......... .......... .......... .......... .......... 52% 342M 0s 2019-12-09T09:43:04.023 INFO:teuthology.orchestra.run.smithi033.stderr: 38450K .......... .......... .......... .......... .......... 52% 399M 0s 2019-12-09T09:43:04.023 INFO:teuthology.orchestra.run.smithi033.stderr: 38500K .......... .......... .......... .......... .......... 52% 314M 0s 2019-12-09T09:43:04.023 INFO:teuthology.orchestra.run.smithi033.stderr: 38550K .......... .......... .......... .......... .......... 52% 291M 0s 2019-12-09T09:43:04.023 INFO:teuthology.orchestra.run.smithi033.stderr: 38600K .......... .......... .......... .......... .......... 52% 358M 0s 2019-12-09T09:43:04.023 INFO:teuthology.orchestra.run.smithi033.stderr: 38650K .......... .......... .......... .......... .......... 52% 298M 0s 2019-12-09T09:43:04.024 INFO:teuthology.orchestra.run.smithi033.stderr: 38700K .......... .......... .......... .......... .......... 52% 337M 0s 2019-12-09T09:43:04.024 INFO:teuthology.orchestra.run.smithi033.stderr: 38750K .......... .......... .......... .......... .......... 52% 332M 0s 2019-12-09T09:43:04.024 INFO:teuthology.orchestra.run.smithi033.stderr: 38800K .......... .......... .......... .......... .......... 52% 377M 0s 2019-12-09T09:43:04.024 INFO:teuthology.orchestra.run.smithi033.stderr: 38850K .......... .......... .......... .......... .......... 52% 412M 0s 2019-12-09T09:43:04.024 INFO:teuthology.orchestra.run.smithi033.stderr: 38900K .......... .......... .......... .......... .......... 52% 408M 0s 2019-12-09T09:43:04.024 INFO:teuthology.orchestra.run.smithi033.stderr: 38950K .......... .......... .......... .......... .......... 53% 382M 0s 2019-12-09T09:43:04.024 INFO:teuthology.orchestra.run.smithi033.stderr: 39000K .......... .......... .......... .......... .......... 53% 335M 0s 2019-12-09T09:43:04.025 INFO:teuthology.orchestra.run.smithi033.stderr: 39050K .......... .......... .......... .......... .......... 53% 395M 0s 2019-12-09T09:43:04.025 INFO:teuthology.orchestra.run.smithi033.stderr: 39100K .......... .......... .......... .......... .......... 53% 380M 0s 2019-12-09T09:43:04.025 INFO:teuthology.orchestra.run.smithi033.stderr: 39150K .......... .......... .......... .......... .......... 53% 307M 0s 2019-12-09T09:43:04.025 INFO:teuthology.orchestra.run.smithi033.stderr: 39200K .......... .......... .......... .......... .......... 53% 307M 0s 2019-12-09T09:43:04.025 INFO:teuthology.orchestra.run.smithi033.stderr: 39250K .......... .......... .......... .......... .......... 53% 281M 0s 2019-12-09T09:43:04.025 INFO:teuthology.orchestra.run.smithi033.stderr: 39300K .......... .......... .......... .......... .......... 53% 371M 0s 2019-12-09T09:43:04.025 INFO:teuthology.orchestra.run.smithi033.stderr: 39350K .......... .......... .......... .......... .......... 53% 267M 0s 2019-12-09T09:43:04.026 INFO:teuthology.orchestra.run.smithi033.stderr: 39400K .......... .......... .......... .......... .......... 53% 322M 0s 2019-12-09T09:43:04.026 INFO:teuthology.orchestra.run.smithi033.stderr: 39450K .......... .......... .......... .......... .......... 53% 288M 0s 2019-12-09T09:43:04.026 INFO:teuthology.orchestra.run.smithi033.stderr: 39500K .......... .......... .......... .......... .......... 53% 273M 0s 2019-12-09T09:43:04.026 INFO:teuthology.orchestra.run.smithi033.stderr: 39550K .......... .......... .......... .......... .......... 53% 288M 0s 2019-12-09T09:43:04.026 INFO:teuthology.orchestra.run.smithi033.stderr: 39600K .......... .......... .......... .......... .......... 53% 296M 0s 2019-12-09T09:43:04.026 INFO:teuthology.orchestra.run.smithi033.stderr: 39650K .......... .......... .......... .......... .......... 53% 287M 0s 2019-12-09T09:43:04.026 INFO:teuthology.orchestra.run.smithi033.stderr: 39700K .......... .......... .......... .......... .......... 54% 354M 0s 2019-12-09T09:43:04.027 INFO:teuthology.orchestra.run.smithi033.stderr: 39750K .......... .......... .......... .......... .......... 54% 368M 0s 2019-12-09T09:43:04.027 INFO:teuthology.orchestra.run.smithi033.stderr: 39800K .......... .......... .......... .......... .......... 54% 411M 0s 2019-12-09T09:43:04.027 INFO:teuthology.orchestra.run.smithi033.stderr: 39850K .......... .......... .......... .......... .......... 54% 391M 0s 2019-12-09T09:43:04.027 INFO:teuthology.orchestra.run.smithi033.stderr: 39900K .......... .......... .......... .......... .......... 54% 420M 0s 2019-12-09T09:43:04.027 INFO:teuthology.orchestra.run.smithi033.stderr: 39950K .......... .......... .......... .......... .......... 54% 331M 0s 2019-12-09T09:43:04.027 INFO:teuthology.orchestra.run.smithi033.stderr: 40000K .......... .......... .......... .......... .......... 54% 293M 0s 2019-12-09T09:43:04.027 INFO:teuthology.orchestra.run.smithi033.stderr: 40050K .......... .......... .......... .......... .......... 54% 345M 0s 2019-12-09T09:43:04.028 INFO:teuthology.orchestra.run.smithi033.stderr: 40100K .......... .......... .......... .......... .......... 54% 359M 0s 2019-12-09T09:43:04.028 INFO:teuthology.orchestra.run.smithi033.stderr: 40150K .......... .......... .......... .......... .......... 54% 287M 0s 2019-12-09T09:43:04.028 INFO:teuthology.orchestra.run.smithi033.stderr: 40200K .......... .......... .......... .......... .......... 54% 380M 0s 2019-12-09T09:43:04.028 INFO:teuthology.orchestra.run.smithi033.stderr: 40250K .......... .......... .......... .......... .......... 54% 371M 0s 2019-12-09T09:43:04.028 INFO:teuthology.orchestra.run.smithi033.stderr: 40300K .......... .......... .......... .......... .......... 54% 347M 0s 2019-12-09T09:43:04.028 INFO:teuthology.orchestra.run.smithi033.stderr: 40350K .......... .......... .......... .......... .......... 54% 329M 0s 2019-12-09T09:43:04.029 INFO:teuthology.orchestra.run.smithi033.stderr: 40400K .......... .......... .......... .......... .......... 54% 376M 0s 2019-12-09T09:43:04.029 INFO:teuthology.orchestra.run.smithi033.stderr: 40450K .......... .......... .......... .......... .......... 55% 338M 0s 2019-12-09T09:43:04.029 INFO:teuthology.orchestra.run.smithi033.stderr: 40500K .......... .......... .......... .......... .......... 55% 309M 0s 2019-12-09T09:43:04.029 INFO:teuthology.orchestra.run.smithi033.stderr: 40550K .......... .......... .......... .......... .......... 55% 304M 0s 2019-12-09T09:43:04.029 INFO:teuthology.orchestra.run.smithi033.stderr: 40600K .......... .......... .......... .......... .......... 55% 372M 0s 2019-12-09T09:43:04.029 INFO:teuthology.orchestra.run.smithi033.stderr: 40650K .......... .......... .......... .......... .......... 55% 367M 0s 2019-12-09T09:43:04.030 INFO:teuthology.orchestra.run.smithi033.stderr: 40700K .......... .......... .......... .......... .......... 55% 364M 0s 2019-12-09T09:43:04.030 INFO:teuthology.orchestra.run.smithi033.stderr: 40750K .......... .......... .......... .......... .......... 55% 340M 0s 2019-12-09T09:43:04.030 INFO:teuthology.orchestra.run.smithi033.stderr: 40800K .......... .......... .......... .......... .......... 55% 353M 0s 2019-12-09T09:43:04.030 INFO:teuthology.orchestra.run.smithi033.stderr: 40850K .......... .......... .......... .......... .......... 55% 362M 0s 2019-12-09T09:43:04.030 INFO:teuthology.orchestra.run.smithi033.stderr: 40900K .......... .......... .......... .......... .......... 55% 383M 0s 2019-12-09T09:43:04.030 INFO:teuthology.orchestra.run.smithi033.stderr: 40950K .......... .......... .......... .......... .......... 55% 360M 0s 2019-12-09T09:43:04.031 INFO:teuthology.orchestra.run.smithi033.stderr: 41000K .......... .......... .......... .......... .......... 55% 358M 0s 2019-12-09T09:43:04.031 INFO:teuthology.orchestra.run.smithi033.stderr: 41050K .......... .......... .......... .......... .......... 55% 378M 0s 2019-12-09T09:43:04.031 INFO:teuthology.orchestra.run.smithi033.stderr: 41100K .......... .......... .......... .......... .......... 55% 430M 0s 2019-12-09T09:43:04.031 INFO:teuthology.orchestra.run.smithi033.stderr: 41150K .......... .......... .......... .......... .......... 56% 309M 0s 2019-12-09T09:43:04.031 INFO:teuthology.orchestra.run.smithi033.stderr: 41200K .......... .......... .......... .......... .......... 56% 366M 0s 2019-12-09T09:43:04.031 INFO:teuthology.orchestra.run.smithi033.stderr: 41250K .......... .......... .......... .......... .......... 56% 336M 0s 2019-12-09T09:43:04.031 INFO:teuthology.orchestra.run.smithi033.stderr: 41300K .......... .......... .......... .......... .......... 56% 359M 0s 2019-12-09T09:43:04.032 INFO:teuthology.orchestra.run.smithi033.stderr: 41350K .......... .......... .......... .......... .......... 56% 311M 0s 2019-12-09T09:43:04.032 INFO:teuthology.orchestra.run.smithi033.stderr: 41400K .......... .......... .......... .......... .......... 56% 404M 0s 2019-12-09T09:43:04.032 INFO:teuthology.orchestra.run.smithi033.stderr: 41450K .......... .......... .......... .......... .......... 56% 437M 0s 2019-12-09T09:43:04.032 INFO:teuthology.orchestra.run.smithi033.stderr: 41500K .......... .......... .......... .......... .......... 56% 351M 0s 2019-12-09T09:43:04.032 INFO:teuthology.orchestra.run.smithi033.stderr: 41550K .......... .......... .......... .......... .......... 56% 348M 0s 2019-12-09T09:43:04.032 INFO:teuthology.orchestra.run.smithi033.stderr: 41600K .......... .......... .......... .......... .......... 56% 358M 0s 2019-12-09T09:43:04.032 INFO:teuthology.orchestra.run.smithi033.stderr: 41650K .......... .......... .......... .......... .......... 56% 375M 0s 2019-12-09T09:43:04.033 INFO:teuthology.orchestra.run.smithi033.stderr: 41700K .......... .......... .......... .......... .......... 56% 431M 0s 2019-12-09T09:43:04.033 INFO:teuthology.orchestra.run.smithi033.stderr: 41750K .......... .......... .......... .......... .......... 56% 368M 0s 2019-12-09T09:43:04.033 INFO:teuthology.orchestra.run.smithi033.stderr: 41800K .......... .......... .......... .......... .......... 56% 348M 0s 2019-12-09T09:43:04.033 INFO:teuthology.orchestra.run.smithi033.stderr: 41850K .......... .......... .......... .......... .......... 56% 336M 0s 2019-12-09T09:43:04.033 INFO:teuthology.orchestra.run.smithi033.stderr: 41900K .......... .......... .......... .......... .......... 57% 386M 0s 2019-12-09T09:43:04.033 INFO:teuthology.orchestra.run.smithi033.stderr: 41950K .......... .......... .......... .......... .......... 57% 246M 0s 2019-12-09T09:43:04.033 INFO:teuthology.orchestra.run.smithi033.stderr: 42000K .......... .......... .......... .......... .......... 57% 386M 0s 2019-12-09T09:43:04.034 INFO:teuthology.orchestra.run.smithi033.stderr: 42050K .......... .......... .......... .......... .......... 57% 320M 0s 2019-12-09T09:43:04.034 INFO:teuthology.orchestra.run.smithi033.stderr: 42100K .......... .......... .......... .......... .......... 57% 310M 0s 2019-12-09T09:43:04.034 INFO:teuthology.orchestra.run.smithi033.stderr: 42150K .......... .......... .......... .......... .......... 57% 268M 0s 2019-12-09T09:43:04.034 INFO:teuthology.orchestra.run.smithi033.stderr: 42200K .......... .......... .......... .......... .......... 57% 325M 0s 2019-12-09T09:43:04.034 INFO:teuthology.orchestra.run.smithi033.stderr: 42250K .......... .......... .......... .......... .......... 57% 319M 0s 2019-12-09T09:43:04.034 INFO:teuthology.orchestra.run.smithi033.stderr: 42300K .......... .......... .......... .......... .......... 57% 354M 0s 2019-12-09T09:43:04.035 INFO:teuthology.orchestra.run.smithi033.stderr: 42350K .......... .......... .......... .......... .......... 57% 290M 0s 2019-12-09T09:43:04.035 INFO:teuthology.orchestra.run.smithi033.stderr: 42400K .......... .......... .......... .......... .......... 57% 416M 0s 2019-12-09T09:43:04.035 INFO:teuthology.orchestra.run.smithi033.stderr: 42450K .......... .......... .......... .......... .......... 57% 287M 0s 2019-12-09T09:43:04.035 INFO:teuthology.orchestra.run.smithi033.stderr: 42500K .......... .......... .......... .......... .......... 57% 345M 0s 2019-12-09T09:43:04.035 INFO:teuthology.orchestra.run.smithi033.stderr: 42550K .......... .......... .......... .......... .......... 57% 308M 0s 2019-12-09T09:43:04.035 INFO:teuthology.orchestra.run.smithi033.stderr: 42600K .......... .......... .......... .......... .......... 57% 347M 0s 2019-12-09T09:43:04.036 INFO:teuthology.orchestra.run.smithi033.stderr: 42650K .......... .......... .......... .......... .......... 58% 303M 0s 2019-12-09T09:43:04.036 INFO:teuthology.orchestra.run.smithi033.stderr: 42700K .......... .......... .......... .......... .......... 58% 350M 0s 2019-12-09T09:43:04.036 INFO:teuthology.orchestra.run.smithi033.stderr: 42750K .......... .......... .......... .......... .......... 58% 298M 0s 2019-12-09T09:43:04.036 INFO:teuthology.orchestra.run.smithi033.stderr: 42800K .......... .......... .......... .......... .......... 58% 344M 0s 2019-12-09T09:43:04.036 INFO:teuthology.orchestra.run.smithi033.stderr: 42850K .......... .......... .......... .......... .......... 58% 368M 0s 2019-12-09T09:43:04.036 INFO:teuthology.orchestra.run.smithi033.stderr: 42900K .......... .......... .......... .......... .......... 58% 382M 0s 2019-12-09T09:43:04.036 INFO:teuthology.orchestra.run.smithi033.stderr: 42950K .......... .......... .......... .......... .......... 58% 225M 0s 2019-12-09T09:43:04.037 INFO:teuthology.orchestra.run.smithi033.stderr: 43000K .......... .......... .......... .......... .......... 58% 348M 0s 2019-12-09T09:43:04.037 INFO:teuthology.orchestra.run.smithi033.stderr: 43050K .......... .......... .......... .......... .......... 58% 339M 0s 2019-12-09T09:43:04.037 INFO:teuthology.orchestra.run.smithi033.stderr: 43100K .......... .......... .......... .......... .......... 58% 217M 0s 2019-12-09T09:43:04.037 INFO:teuthology.orchestra.run.smithi033.stderr: 43150K .......... .......... .......... .......... .......... 58% 209M 0s 2019-12-09T09:43:04.037 INFO:teuthology.orchestra.run.smithi033.stderr: 43200K .......... .......... .......... .......... .......... 58% 302M 0s 2019-12-09T09:43:04.037 INFO:teuthology.orchestra.run.smithi033.stderr: 43250K .......... .......... .......... .......... .......... 58% 350M 0s 2019-12-09T09:43:04.037 INFO:teuthology.orchestra.run.smithi033.stderr: 43300K .......... .......... .......... .......... .......... 58% 355M 0s 2019-12-09T09:43:04.038 INFO:teuthology.orchestra.run.smithi033.stderr: 43350K .......... .......... .......... .......... .......... 59% 294M 0s 2019-12-09T09:43:04.038 INFO:teuthology.orchestra.run.smithi033.stderr: 43400K .......... .......... .......... .......... .......... 59% 350M 0s 2019-12-09T09:43:04.038 INFO:teuthology.orchestra.run.smithi033.stderr: 43450K .......... .......... .......... .......... .......... 59% 336M 0s 2019-12-09T09:43:04.038 INFO:teuthology.orchestra.run.smithi033.stderr: 43500K .......... .......... .......... .......... .......... 59% 379M 0s 2019-12-09T09:43:04.038 INFO:teuthology.orchestra.run.smithi033.stderr: 43550K .......... .......... .......... .......... .......... 59% 280M 0s 2019-12-09T09:43:04.038 INFO:teuthology.orchestra.run.smithi033.stderr: 43600K .......... .......... .......... .......... .......... 59% 359M 0s 2019-12-09T09:43:04.038 INFO:teuthology.orchestra.run.smithi033.stderr: 43650K .......... .......... .......... .......... .......... 59% 383M 0s 2019-12-09T09:43:04.039 INFO:teuthology.orchestra.run.smithi033.stderr: 43700K .......... .......... .......... .......... .......... 59% 330M 0s 2019-12-09T09:43:04.039 INFO:teuthology.orchestra.run.smithi033.stderr: 43750K .......... .......... .......... .......... .......... 59% 351M 0s 2019-12-09T09:43:04.039 INFO:teuthology.orchestra.run.smithi033.stderr: 43800K .......... .......... .......... .......... .......... 59% 286M 0s 2019-12-09T09:43:04.039 INFO:teuthology.orchestra.run.smithi033.stderr: 43850K .......... .......... .......... .......... .......... 59% 254M 0s 2019-12-09T09:43:04.039 INFO:teuthology.orchestra.run.smithi033.stderr: 43900K .......... .......... .......... .......... .......... 59% 257M 0s 2019-12-09T09:43:04.039 INFO:teuthology.orchestra.run.smithi033.stderr: 43950K .......... .......... .......... .......... .......... 59% 376M 0s 2019-12-09T09:43:04.039 INFO:teuthology.orchestra.run.smithi033.stderr: 44000K .......... .......... .......... .......... .......... 59% 376M 0s 2019-12-09T09:43:04.040 INFO:teuthology.orchestra.run.smithi033.stderr: 44050K .......... .......... .......... .......... .......... 59% 381M 0s 2019-12-09T09:43:04.040 INFO:teuthology.orchestra.run.smithi033.stderr: 44100K .......... .......... .......... .......... .......... 60% 345M 0s 2019-12-09T09:43:04.040 INFO:teuthology.orchestra.run.smithi033.stderr: 44150K .......... .......... .......... .......... .......... 60% 332M 0s 2019-12-09T09:43:04.040 INFO:teuthology.orchestra.run.smithi033.stderr: 44200K .......... .......... .......... .......... .......... 60% 336M 0s 2019-12-09T09:43:04.040 INFO:teuthology.orchestra.run.smithi033.stderr: 44250K .......... .......... .......... .......... .......... 60% 348M 0s 2019-12-09T09:43:04.040 INFO:teuthology.orchestra.run.smithi033.stderr: 44300K .......... .......... .......... .......... .......... 60% 327M 0s 2019-12-09T09:43:04.040 INFO:teuthology.orchestra.run.smithi033.stderr: 44350K .......... .......... .......... .......... .......... 60% 295M 0s 2019-12-09T09:43:04.041 INFO:teuthology.orchestra.run.smithi033.stderr: 44400K .......... .......... .......... .......... .......... 60% 329M 0s 2019-12-09T09:43:04.041 INFO:teuthology.orchestra.run.smithi033.stderr: 44450K .......... .......... .......... .......... .......... 60% 423M 0s 2019-12-09T09:43:04.041 INFO:teuthology.orchestra.run.smithi033.stderr: 44500K .......... .......... .......... .......... .......... 60% 369M 0s 2019-12-09T09:43:04.041 INFO:teuthology.orchestra.run.smithi033.stderr: 44550K .......... .......... .......... .......... .......... 60% 280M 0s 2019-12-09T09:43:04.041 INFO:teuthology.orchestra.run.smithi033.stderr: 44600K .......... .......... .......... .......... .......... 60% 333M 0s 2019-12-09T09:43:04.041 INFO:teuthology.orchestra.run.smithi033.stderr: 44650K .......... .......... .......... .......... .......... 60% 393M 0s 2019-12-09T09:43:04.041 INFO:teuthology.orchestra.run.smithi033.stderr: 44700K .......... .......... .......... .......... .......... 60% 368M 0s 2019-12-09T09:43:04.042 INFO:teuthology.orchestra.run.smithi033.stderr: 44750K .......... .......... .......... .......... .......... 60% 316M 0s 2019-12-09T09:43:04.042 INFO:teuthology.orchestra.run.smithi033.stderr: 44800K .......... .......... .......... .......... .......... 60% 389M 0s 2019-12-09T09:43:04.042 INFO:teuthology.orchestra.run.smithi033.stderr: 44850K .......... .......... .......... .......... .......... 61% 395M 0s 2019-12-09T09:43:04.042 INFO:teuthology.orchestra.run.smithi033.stderr: 44900K .......... .......... .......... .......... .......... 61% 397M 0s 2019-12-09T09:43:04.042 INFO:teuthology.orchestra.run.smithi033.stderr: 44950K .......... .......... .......... .......... .......... 61% 337M 0s 2019-12-09T09:43:04.042 INFO:teuthology.orchestra.run.smithi033.stderr: 45000K .......... .......... .......... .......... .......... 61% 390M 0s 2019-12-09T09:43:04.042 INFO:teuthology.orchestra.run.smithi033.stderr: 45050K .......... .......... .......... .......... .......... 61% 387M 0s 2019-12-09T09:43:04.043 INFO:teuthology.orchestra.run.smithi033.stderr: 45100K .......... .......... .......... .......... .......... 61% 364M 0s 2019-12-09T09:43:04.043 INFO:teuthology.orchestra.run.smithi033.stderr: 45150K .......... .......... .......... .......... .......... 61% 325M 0s 2019-12-09T09:43:04.043 INFO:teuthology.orchestra.run.smithi033.stderr: 45200K .......... .......... .......... .......... .......... 61% 403M 0s 2019-12-09T09:43:04.043 INFO:teuthology.orchestra.run.smithi033.stderr: 45250K .......... .......... .......... .......... .......... 61% 394M 0s 2019-12-09T09:43:04.043 INFO:teuthology.orchestra.run.smithi033.stderr: 45300K .......... .......... .......... .......... .......... 61% 392M 0s 2019-12-09T09:43:04.043 INFO:teuthology.orchestra.run.smithi033.stderr: 45350K .......... .......... .......... .......... .......... 61% 261M 0s 2019-12-09T09:43:04.043 INFO:teuthology.orchestra.run.smithi033.stderr: 45400K .......... .......... .......... .......... .......... 61% 364M 0s 2019-12-09T09:43:04.044 INFO:teuthology.orchestra.run.smithi033.stderr: 45450K .......... .......... .......... .......... .......... 61% 337M 0s 2019-12-09T09:43:04.044 INFO:teuthology.orchestra.run.smithi033.stderr: 45500K .......... .......... .......... .......... .......... 61% 303M 0s 2019-12-09T09:43:04.044 INFO:teuthology.orchestra.run.smithi033.stderr: 45550K .......... .......... .......... .......... .......... 61% 384M 0s 2019-12-09T09:43:04.044 INFO:teuthology.orchestra.run.smithi033.stderr: 45600K .......... .......... .......... .......... .......... 62% 323M 0s 2019-12-09T09:43:04.044 INFO:teuthology.orchestra.run.smithi033.stderr: 45650K .......... .......... .......... .......... .......... 62% 370M 0s 2019-12-09T09:43:04.044 INFO:teuthology.orchestra.run.smithi033.stderr: 45700K .......... .......... .......... .......... .......... 62% 373M 0s 2019-12-09T09:43:04.044 INFO:teuthology.orchestra.run.smithi033.stderr: 45750K .......... .......... .......... .......... .......... 62% 323M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 45800K .......... .......... .......... .......... .......... 62% 390M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 45850K .......... .......... .......... .......... .......... 62% 340M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 45900K .......... .......... .......... .......... .......... 62% 338M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 45950K .......... .......... .......... .......... .......... 62% 218M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 46000K .......... .......... .......... .......... .......... 62% 172M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 46050K .......... .......... .......... .......... .......... 62% 204M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 46100K .......... .......... .......... .......... .......... 62% 231M 0s 2019-12-09T09:43:04.045 INFO:teuthology.orchestra.run.smithi033.stderr: 46150K .......... .......... .......... .......... .......... 62% 218M 0s 2019-12-09T09:43:04.046 INFO:teuthology.orchestra.run.smithi033.stderr: 46200K .......... .......... .......... .......... .......... 62% 215M 0s 2019-12-09T09:43:04.046 INFO:teuthology.orchestra.run.smithi033.stderr: 46250K .......... .......... .......... .......... .......... 62% 303M 0s 2019-12-09T09:43:04.046 INFO:teuthology.orchestra.run.smithi033.stderr: 46300K .......... .......... .......... .......... .......... 63% 240M 0s 2019-12-09T09:43:04.046 INFO:teuthology.orchestra.run.smithi033.stderr: 46350K .......... .......... .......... .......... .......... 63% 192M 0s 2019-12-09T09:43:04.046 INFO:teuthology.orchestra.run.smithi033.stderr: 46400K .......... .......... .......... .......... .......... 63% 239M 0s 2019-12-09T09:43:04.046 INFO:teuthology.orchestra.run.smithi033.stderr: 46450K .......... .......... .......... .......... .......... 63% 215M 0s 2019-12-09T09:43:04.046 INFO:teuthology.orchestra.run.smithi033.stderr: 46500K .......... .......... .......... .......... .......... 63% 219M 0s 2019-12-09T09:43:04.047 INFO:teuthology.orchestra.run.smithi033.stderr: 46550K .......... .......... .......... .......... .......... 63% 200M 0s 2019-12-09T09:43:04.047 INFO:teuthology.orchestra.run.smithi033.stderr: 46600K .......... .......... .......... .......... .......... 63% 215M 0s 2019-12-09T09:43:04.047 INFO:teuthology.orchestra.run.smithi033.stderr: 46650K .......... .......... .......... .......... .......... 63% 267M 0s 2019-12-09T09:43:04.047 INFO:teuthology.orchestra.run.smithi033.stderr: 46700K .......... .......... .......... .......... .......... 63% 361M 0s 2019-12-09T09:43:04.047 INFO:teuthology.orchestra.run.smithi033.stderr: 46750K .......... .......... .......... .......... .......... 63% 310M 0s 2019-12-09T09:43:04.047 INFO:teuthology.orchestra.run.smithi033.stderr: 46800K .......... .......... .......... .......... .......... 63% 377M 0s 2019-12-09T09:43:04.047 INFO:teuthology.orchestra.run.smithi033.stderr: 46850K .......... .......... .......... .......... .......... 63% 348M 0s 2019-12-09T09:43:04.048 INFO:teuthology.orchestra.run.smithi033.stderr: 46900K .......... .......... .......... .......... .......... 63% 314M 0s 2019-12-09T09:43:04.048 INFO:teuthology.orchestra.run.smithi033.stderr: 46950K .......... .......... .......... .......... .......... 63% 325M 0s 2019-12-09T09:43:04.048 INFO:teuthology.orchestra.run.smithi033.stderr: 47000K .......... .......... .......... .......... .......... 63% 296M 0s 2019-12-09T09:43:04.048 INFO:teuthology.orchestra.run.smithi033.stderr: 47050K .......... .......... .......... .......... .......... 64% 349M 0s 2019-12-09T09:43:04.048 INFO:teuthology.orchestra.run.smithi033.stderr: 47100K .......... .......... .......... .......... .......... 64% 339M 0s 2019-12-09T09:43:04.048 INFO:teuthology.orchestra.run.smithi033.stderr: 47150K .......... .......... .......... .......... .......... 64% 328M 0s 2019-12-09T09:43:04.048 INFO:teuthology.orchestra.run.smithi033.stderr: 47200K .......... .......... .......... .......... .......... 64% 340M 0s 2019-12-09T09:43:04.049 INFO:teuthology.orchestra.run.smithi033.stderr: 47250K .......... .......... .......... .......... .......... 64% 311M 0s 2019-12-09T09:43:04.049 INFO:teuthology.orchestra.run.smithi033.stderr: 47300K .......... .......... .......... .......... .......... 64% 314M 0s 2019-12-09T09:43:04.049 INFO:teuthology.orchestra.run.smithi033.stderr: 47350K .......... .......... .......... .......... .......... 64% 246M 0s 2019-12-09T09:43:04.049 INFO:teuthology.orchestra.run.smithi033.stderr: 47400K .......... .......... .......... .......... .......... 64% 326M 0s 2019-12-09T09:43:04.049 INFO:teuthology.orchestra.run.smithi033.stderr: 47450K .......... .......... .......... .......... .......... 64% 289M 0s 2019-12-09T09:43:04.049 INFO:teuthology.orchestra.run.smithi033.stderr: 47500K .......... .......... .......... .......... .......... 64% 348M 0s 2019-12-09T09:43:04.049 INFO:teuthology.orchestra.run.smithi033.stderr: 47550K .......... .......... .......... .......... .......... 64% 280M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47600K .......... .......... .......... .......... .......... 64% 271M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47650K .......... .......... .......... .......... .......... 64% 299M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47700K .......... .......... .......... .......... .......... 64% 411M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47750K .......... .......... .......... .......... .......... 64% 244M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47800K .......... .......... .......... .......... .......... 65% 283M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47850K .......... .......... .......... .......... .......... 65% 346M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47900K .......... .......... .......... .......... .......... 65% 346M 0s 2019-12-09T09:43:04.050 INFO:teuthology.orchestra.run.smithi033.stderr: 47950K .......... .......... .......... .......... .......... 65% 262M 0s 2019-12-09T09:43:04.051 INFO:teuthology.orchestra.run.smithi033.stderr: 48000K .......... .......... .......... .......... .......... 65% 330M 0s 2019-12-09T09:43:04.051 INFO:teuthology.orchestra.run.smithi033.stderr: 48050K .......... .......... .......... .......... .......... 65% 350M 0s 2019-12-09T09:43:04.051 INFO:teuthology.orchestra.run.smithi033.stderr: 48100K .......... .......... .......... .......... .......... 65% 360M 0s 2019-12-09T09:43:04.051 INFO:teuthology.orchestra.run.smithi033.stderr: 48150K .......... .......... .......... .......... .......... 65% 287M 0s 2019-12-09T09:43:04.051 INFO:teuthology.orchestra.run.smithi033.stderr: 48200K .......... .......... .......... .......... .......... 65% 346M 0s 2019-12-09T09:43:04.051 INFO:teuthology.orchestra.run.smithi033.stderr: 48250K .......... .......... .......... .......... .......... 65% 324M 0s 2019-12-09T09:43:04.051 INFO:teuthology.orchestra.run.smithi033.stderr: 48300K .......... .......... .......... .......... .......... 65% 332M 0s 2019-12-09T09:43:04.052 INFO:teuthology.orchestra.run.smithi033.stderr: 48350K .......... .......... .......... .......... .......... 65% 296M 0s 2019-12-09T09:43:04.052 INFO:teuthology.orchestra.run.smithi033.stderr: 48400K .......... .......... .......... .......... .......... 65% 341M 0s 2019-12-09T09:43:04.052 INFO:teuthology.orchestra.run.smithi033.stderr: 48450K .......... .......... .......... .......... .......... 65% 372M 0s 2019-12-09T09:43:04.052 INFO:teuthology.orchestra.run.smithi033.stderr: 48500K .......... .......... .......... .......... .......... 66% 368M 0s 2019-12-09T09:43:04.052 INFO:teuthology.orchestra.run.smithi033.stderr: 48550K .......... .......... .......... .......... .......... 66% 296M 0s 2019-12-09T09:43:04.052 INFO:teuthology.orchestra.run.smithi033.stderr: 48600K .......... .......... .......... .......... .......... 66% 260M 0s 2019-12-09T09:43:04.052 INFO:teuthology.orchestra.run.smithi033.stderr: 48650K .......... .......... .......... .......... .......... 66% 378M 0s 2019-12-09T09:43:04.053 INFO:teuthology.orchestra.run.smithi033.stderr: 48700K .......... .......... .......... .......... .......... 66% 390M 0s 2019-12-09T09:43:04.053 INFO:teuthology.orchestra.run.smithi033.stderr: 48750K .......... .......... .......... .......... .......... 66% 252M 0s 2019-12-09T09:43:04.053 INFO:teuthology.orchestra.run.smithi033.stderr: 48800K .......... .......... .......... .......... .......... 66% 318M 0s 2019-12-09T09:43:04.053 INFO:teuthology.orchestra.run.smithi033.stderr: 48850K .......... .......... .......... .......... .......... 66% 315M 0s 2019-12-09T09:43:04.053 INFO:teuthology.orchestra.run.smithi033.stderr: 48900K .......... .......... .......... .......... .......... 66% 386M 0s 2019-12-09T09:43:04.053 INFO:teuthology.orchestra.run.smithi033.stderr: 48950K .......... .......... .......... .......... .......... 66% 338M 0s 2019-12-09T09:43:04.053 INFO:teuthology.orchestra.run.smithi033.stderr: 49000K .......... .......... .......... .......... .......... 66% 309M 0s 2019-12-09T09:43:04.054 INFO:teuthology.orchestra.run.smithi033.stderr: 49050K .......... .......... .......... .......... .......... 66% 352M 0s 2019-12-09T09:43:04.054 INFO:teuthology.orchestra.run.smithi033.stderr: 49100K .......... .......... .......... .......... .......... 66% 371M 0s 2019-12-09T09:43:04.054 INFO:teuthology.orchestra.run.smithi033.stderr: 49150K .......... .......... .......... .......... .......... 66% 289M 0s 2019-12-09T09:43:04.054 INFO:teuthology.orchestra.run.smithi033.stderr: 49200K .......... .......... .......... .......... .......... 66% 353M 0s 2019-12-09T09:43:04.054 INFO:teuthology.orchestra.run.smithi033.stderr: 49250K .......... .......... .......... .......... .......... 67% 359M 0s 2019-12-09T09:43:04.054 INFO:teuthology.orchestra.run.smithi033.stderr: 49300K .......... .......... .......... .......... .......... 67% 372M 0s 2019-12-09T09:43:04.054 INFO:teuthology.orchestra.run.smithi033.stderr: 49350K .......... .......... .......... .......... .......... 67% 298M 0s 2019-12-09T09:43:04.055 INFO:teuthology.orchestra.run.smithi033.stderr: 49400K .......... .......... .......... .......... .......... 67% 309M 0s 2019-12-09T09:43:04.055 INFO:teuthology.orchestra.run.smithi033.stderr: 49450K .......... .......... .......... .......... .......... 67% 343M 0s 2019-12-09T09:43:04.055 INFO:teuthology.orchestra.run.smithi033.stderr: 49500K .......... .......... .......... .......... .......... 67% 316M 0s 2019-12-09T09:43:04.055 INFO:teuthology.orchestra.run.smithi033.stderr: 49550K .......... .......... .......... .......... .......... 67% 299M 0s 2019-12-09T09:43:04.055 INFO:teuthology.orchestra.run.smithi033.stderr: 49600K .......... .......... .......... .......... .......... 67% 330M 0s 2019-12-09T09:43:04.055 INFO:teuthology.orchestra.run.smithi033.stderr: 49650K .......... .......... .......... .......... .......... 67% 332M 0s 2019-12-09T09:43:04.055 INFO:teuthology.orchestra.run.smithi033.stderr: 49700K .......... .......... .......... .......... .......... 67% 328M 0s 2019-12-09T09:43:04.056 INFO:teuthology.orchestra.run.smithi033.stderr: 49750K .......... .......... .......... .......... .......... 67% 231M 0s 2019-12-09T09:43:04.056 INFO:teuthology.orchestra.run.smithi033.stderr: 49800K .......... .......... .......... .......... .......... 67% 321M 0s 2019-12-09T09:43:04.056 INFO:teuthology.orchestra.run.smithi033.stderr: 49850K .......... .......... .......... .......... .......... 67% 305M 0s 2019-12-09T09:43:04.056 INFO:teuthology.orchestra.run.smithi033.stderr: 49900K .......... .......... .......... .......... .......... 67% 363M 0s 2019-12-09T09:43:04.056 INFO:teuthology.orchestra.run.smithi033.stderr: 49950K .......... .......... .......... .......... .......... 67% 293M 0s 2019-12-09T09:43:04.056 INFO:teuthology.orchestra.run.smithi033.stderr: 50000K .......... .......... .......... .......... .......... 68% 275M 0s 2019-12-09T09:43:04.057 INFO:teuthology.orchestra.run.smithi033.stderr: 50050K .......... .......... .......... .......... .......... 68% 262M 0s 2019-12-09T09:43:04.057 INFO:teuthology.orchestra.run.smithi033.stderr: 50100K .......... .......... .......... .......... .......... 68% 350M 0s 2019-12-09T09:43:04.057 INFO:teuthology.orchestra.run.smithi033.stderr: 50150K .......... .......... .......... .......... .......... 68% 353M 0s 2019-12-09T09:43:04.057 INFO:teuthology.orchestra.run.smithi033.stderr: 50200K .......... .......... .......... .......... .......... 68% 303M 0s 2019-12-09T09:43:04.057 INFO:teuthology.orchestra.run.smithi033.stderr: 50250K .......... .......... .......... .......... .......... 68% 263M 0s 2019-12-09T09:43:04.057 INFO:teuthology.orchestra.run.smithi033.stderr: 50300K .......... .......... .......... .......... .......... 68% 273M 0s 2019-12-09T09:43:04.057 INFO:teuthology.orchestra.run.smithi033.stderr: 50350K .......... .......... .......... .......... .......... 68% 321M 0s 2019-12-09T09:43:04.058 INFO:teuthology.orchestra.run.smithi033.stderr: 50400K .......... .......... .......... .......... .......... 68% 328M 0s 2019-12-09T09:43:04.058 INFO:teuthology.orchestra.run.smithi033.stderr: 50450K .......... .......... .......... .......... .......... 68% 379M 0s 2019-12-09T09:43:04.058 INFO:teuthology.orchestra.run.smithi033.stderr: 50500K .......... .......... .......... .......... .......... 68% 363M 0s 2019-12-09T09:43:04.058 INFO:teuthology.orchestra.run.smithi033.stderr: 50550K .......... .......... .......... .......... .......... 68% 290M 0s 2019-12-09T09:43:04.058 INFO:teuthology.orchestra.run.smithi033.stderr: 50600K .......... .......... .......... .......... .......... 68% 309M 0s 2019-12-09T09:43:04.058 INFO:teuthology.orchestra.run.smithi033.stderr: 50650K .......... .......... .......... .......... .......... 68% 348M 0s 2019-12-09T09:43:04.058 INFO:teuthology.orchestra.run.smithi033.stderr: 50700K .......... .......... .......... .......... .......... 68% 330M 0s 2019-12-09T09:43:04.059 INFO:teuthology.orchestra.run.smithi033.stderr: 50750K .......... .......... .......... .......... .......... 69% 270M 0s 2019-12-09T09:43:04.059 INFO:teuthology.orchestra.run.smithi033.stderr: 50800K .......... .......... .......... .......... .......... 69% 370M 0s 2019-12-09T09:43:04.059 INFO:teuthology.orchestra.run.smithi033.stderr: 50850K .......... .......... .......... .......... .......... 69% 390M 0s 2019-12-09T09:43:04.059 INFO:teuthology.orchestra.run.smithi033.stderr: 50900K .......... .......... .......... .......... .......... 69% 329M 0s 2019-12-09T09:43:04.059 INFO:teuthology.orchestra.run.smithi033.stderr: 50950K .......... .......... .......... .......... .......... 69% 196M 0s 2019-12-09T09:43:04.059 INFO:teuthology.orchestra.run.smithi033.stderr: 51000K .......... .......... .......... .......... .......... 69% 230M 0s 2019-12-09T09:43:04.059 INFO:teuthology.orchestra.run.smithi033.stderr: 51050K .......... .......... .......... .......... .......... 69% 258M 0s 2019-12-09T09:43:04.060 INFO:teuthology.orchestra.run.smithi033.stderr: 51100K .......... .......... .......... .......... .......... 69% 271M 0s 2019-12-09T09:43:04.060 INFO:teuthology.orchestra.run.smithi033.stderr: 51150K .......... .......... .......... .......... .......... 69% 207M 0s 2019-12-09T09:43:04.060 INFO:teuthology.orchestra.run.smithi033.stderr: 51200K .......... .......... .......... .......... .......... 69% 252M 0s 2019-12-09T09:43:04.060 INFO:teuthology.orchestra.run.smithi033.stderr: 51250K .......... .......... .......... .......... .......... 69% 277M 0s 2019-12-09T09:43:04.060 INFO:teuthology.orchestra.run.smithi033.stderr: 51300K .......... .......... .......... .......... .......... 69% 226M 0s 2019-12-09T09:43:04.060 INFO:teuthology.orchestra.run.smithi033.stderr: 51350K .......... .......... .......... .......... .......... 69% 174M 0s 2019-12-09T09:43:04.060 INFO:teuthology.orchestra.run.smithi033.stderr: 51400K .......... .......... .......... .......... .......... 69% 225M 0s 2019-12-09T09:43:04.061 INFO:teuthology.orchestra.run.smithi033.stderr: 51450K .......... .......... .......... .......... .......... 70% 232M 0s 2019-12-09T09:43:04.061 INFO:teuthology.orchestra.run.smithi033.stderr: 51500K .......... .......... .......... .......... .......... 70% 222M 0s 2019-12-09T09:43:04.061 INFO:teuthology.orchestra.run.smithi033.stderr: 51550K .......... .......... .......... .......... .......... 70% 207M 0s 2019-12-09T09:43:04.061 INFO:teuthology.orchestra.run.smithi033.stderr: 51600K .......... .......... .......... .......... .......... 70% 172M 0s 2019-12-09T09:43:04.061 INFO:teuthology.orchestra.run.smithi033.stderr: 51650K .......... .......... .......... .......... .......... 70% 223M 0s 2019-12-09T09:43:04.061 INFO:teuthology.orchestra.run.smithi033.stderr: 51700K .......... .......... .......... .......... .......... 70% 243M 0s 2019-12-09T09:43:04.061 INFO:teuthology.orchestra.run.smithi033.stderr: 51750K .......... .......... .......... .......... .......... 70% 230M 0s 2019-12-09T09:43:04.062 INFO:teuthology.orchestra.run.smithi033.stderr: 51800K .......... .......... .......... .......... .......... 70% 260M 0s 2019-12-09T09:43:04.062 INFO:teuthology.orchestra.run.smithi033.stderr: 51850K .......... .......... .......... .......... .......... 70% 226M 0s 2019-12-09T09:43:04.062 INFO:teuthology.orchestra.run.smithi033.stderr: 51900K .......... .......... .......... .......... .......... 70% 248M 0s 2019-12-09T09:43:04.062 INFO:teuthology.orchestra.run.smithi033.stderr: 51950K .......... .......... .......... .......... .......... 70% 192M 0s 2019-12-09T09:43:04.062 INFO:teuthology.orchestra.run.smithi033.stderr: 52000K .......... .......... .......... .......... .......... 70% 234M 0s 2019-12-09T09:43:04.062 INFO:teuthology.orchestra.run.smithi033.stderr: 52050K .......... .......... .......... .......... .......... 70% 233M 0s 2019-12-09T09:43:04.062 INFO:teuthology.orchestra.run.smithi033.stderr: 52100K .......... .......... .......... .......... .......... 70% 252M 0s 2019-12-09T09:43:04.063 INFO:teuthology.orchestra.run.smithi033.stderr: 52150K .......... .......... .......... .......... .......... 70% 217M 0s 2019-12-09T09:43:04.063 INFO:teuthology.orchestra.run.smithi033.stderr: 52200K .......... .......... .......... .......... .......... 71% 252M 0s 2019-12-09T09:43:04.063 INFO:teuthology.orchestra.run.smithi033.stderr: 52250K .......... .......... .......... .......... .......... 71% 251M 0s 2019-12-09T09:43:04.063 INFO:teuthology.orchestra.run.smithi033.stderr: 52300K .......... .......... .......... .......... .......... 71% 240M 0s 2019-12-09T09:43:04.063 INFO:teuthology.orchestra.run.smithi033.stderr: 52350K .......... .......... .......... .......... .......... 71% 204M 0s 2019-12-09T09:43:04.063 INFO:teuthology.orchestra.run.smithi033.stderr: 52400K .......... .......... .......... .......... .......... 71% 267M 0s 2019-12-09T09:43:04.064 INFO:teuthology.orchestra.run.smithi033.stderr: 52450K .......... .......... .......... .......... .......... 71% 249M 0s 2019-12-09T09:43:04.064 INFO:teuthology.orchestra.run.smithi033.stderr: 52500K .......... .......... .......... .......... .......... 71% 260M 0s 2019-12-09T09:43:04.064 INFO:teuthology.orchestra.run.smithi033.stderr: 52550K .......... .......... .......... .......... .......... 71% 202M 0s 2019-12-09T09:43:04.064 INFO:teuthology.orchestra.run.smithi033.stderr: 52600K .......... .......... .......... .......... .......... 71% 252M 0s 2019-12-09T09:43:04.064 INFO:teuthology.orchestra.run.smithi033.stderr: 52650K .......... .......... .......... .......... .......... 71% 246M 0s 2019-12-09T09:43:04.064 INFO:teuthology.orchestra.run.smithi033.stderr: 52700K .......... .......... .......... .......... .......... 71% 244M 0s 2019-12-09T09:43:04.065 INFO:teuthology.orchestra.run.smithi033.stderr: 52750K .......... .......... .......... .......... .......... 71% 207M 0s 2019-12-09T09:43:04.065 INFO:teuthology.orchestra.run.smithi033.stderr: 52800K .......... .......... .......... .......... .......... 71% 253M 0s 2019-12-09T09:43:04.065 INFO:teuthology.orchestra.run.smithi033.stderr: 52850K .......... .......... .......... .......... .......... 71% 284M 0s 2019-12-09T09:43:04.065 INFO:teuthology.orchestra.run.smithi033.stderr: 52900K .......... .......... .......... .......... .......... 71% 257M 0s 2019-12-09T09:43:04.065 INFO:teuthology.orchestra.run.smithi033.stderr: 52950K .......... .......... .......... .......... .......... 72% 170M 0s 2019-12-09T09:43:04.065 INFO:teuthology.orchestra.run.smithi033.stderr: 53000K .......... .......... .......... .......... .......... 72% 264M 0s 2019-12-09T09:43:04.065 INFO:teuthology.orchestra.run.smithi033.stderr: 53050K .......... .......... .......... .......... .......... 72% 217M 0s 2019-12-09T09:43:04.066 INFO:teuthology.orchestra.run.smithi033.stderr: 53100K .......... .......... .......... .......... .......... 72% 269M 0s 2019-12-09T09:43:04.066 INFO:teuthology.orchestra.run.smithi033.stderr: 53150K .......... .......... .......... .......... .......... 72% 203M 0s 2019-12-09T09:43:04.066 INFO:teuthology.orchestra.run.smithi033.stderr: 53200K .......... .......... .......... .......... .......... 72% 239M 0s 2019-12-09T09:43:04.066 INFO:teuthology.orchestra.run.smithi033.stderr: 53250K .......... .......... .......... .......... .......... 72% 252M 0s 2019-12-09T09:43:04.066 INFO:teuthology.orchestra.run.smithi033.stderr: 53300K .......... .......... .......... .......... .......... 72% 238M 0s 2019-12-09T09:43:04.066 INFO:teuthology.orchestra.run.smithi033.stderr: 53350K .......... .......... .......... .......... .......... 72% 234M 0s 2019-12-09T09:43:04.066 INFO:teuthology.orchestra.run.smithi033.stderr: 53400K .......... .......... .......... .......... .......... 72% 246M 0s 2019-12-09T09:43:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 53450K .......... .......... .......... .......... .......... 72% 255M 0s 2019-12-09T09:43:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 53500K .......... .......... .......... .......... .......... 72% 222M 0s 2019-12-09T09:43:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 53550K .......... .......... .......... .......... .......... 72% 192M 0s 2019-12-09T09:43:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 53600K .......... .......... .......... .......... .......... 72% 259M 0s 2019-12-09T09:43:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 53650K .......... .......... .......... .......... .......... 73% 252M 0s 2019-12-09T09:43:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 53700K .......... .......... .......... .......... .......... 73% 234M 0s 2019-12-09T09:43:04.067 INFO:teuthology.orchestra.run.smithi033.stderr: 53750K .......... .......... .......... .......... .......... 73% 207M 0s 2019-12-09T09:43:04.068 INFO:teuthology.orchestra.run.smithi033.stderr: 53800K .......... .......... .......... .......... .......... 73% 220M 0s 2019-12-09T09:43:04.068 INFO:teuthology.orchestra.run.smithi033.stderr: 53850K .......... .......... .......... .......... .......... 73% 247M 0s 2019-12-09T09:43:04.068 INFO:teuthology.orchestra.run.smithi033.stderr: 53900K .......... .......... .......... .......... .......... 73% 218M 0s 2019-12-09T09:43:04.068 INFO:teuthology.orchestra.run.smithi033.stderr: 53950K .......... .......... .......... .......... .......... 73% 176M 0s 2019-12-09T09:43:04.068 INFO:teuthology.orchestra.run.smithi033.stderr: 54000K .......... .......... .......... .......... .......... 73% 236M 0s 2019-12-09T09:43:04.068 INFO:teuthology.orchestra.run.smithi033.stderr: 54050K .......... .......... .......... .......... .......... 73% 198M 0s 2019-12-09T09:43:04.068 INFO:teuthology.orchestra.run.smithi033.stderr: 54100K .......... .......... .......... .......... .......... 73% 224M 0s 2019-12-09T09:43:04.069 INFO:teuthology.orchestra.run.smithi033.stderr: 54150K .......... .......... .......... .......... .......... 73% 207M 0s 2019-12-09T09:43:04.069 INFO:teuthology.orchestra.run.smithi033.stderr: 54200K .......... .......... .......... .......... .......... 73% 258M 0s 2019-12-09T09:43:04.069 INFO:teuthology.orchestra.run.smithi033.stderr: 54250K .......... .......... .......... .......... .......... 73% 288M 0s 2019-12-09T09:43:04.069 INFO:teuthology.orchestra.run.smithi033.stderr: 54300K .......... .......... .......... .......... .......... 73% 249M 0s 2019-12-09T09:43:04.069 INFO:teuthology.orchestra.run.smithi033.stderr: 54350K .......... .......... .......... .......... .......... 73% 213M 0s 2019-12-09T09:43:04.069 INFO:teuthology.orchestra.run.smithi033.stderr: 54400K .......... .......... .......... .......... .......... 74% 255M 0s 2019-12-09T09:43:04.069 INFO:teuthology.orchestra.run.smithi033.stderr: 54450K .......... .......... .......... .......... .......... 74% 253M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54500K .......... .......... .......... .......... .......... 74% 250M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54550K .......... .......... .......... .......... .......... 74% 225M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54600K .......... .......... .......... .......... .......... 74% 266M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54650K .......... .......... .......... .......... .......... 74% 256M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54700K .......... .......... .......... .......... .......... 74% 208M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54750K .......... .......... .......... .......... .......... 74% 189M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54800K .......... .......... .......... .......... .......... 74% 250M 0s 2019-12-09T09:43:04.070 INFO:teuthology.orchestra.run.smithi033.stderr: 54850K .......... .......... .......... .......... .......... 74% 253M 0s 2019-12-09T09:43:04.071 INFO:teuthology.orchestra.run.smithi033.stderr: 54900K .......... .......... .......... .......... .......... 74% 237M 0s 2019-12-09T09:43:04.071 INFO:teuthology.orchestra.run.smithi033.stderr: 54950K .......... .......... .......... .......... .......... 74% 232M 0s 2019-12-09T09:43:04.071 INFO:teuthology.orchestra.run.smithi033.stderr: 55000K .......... .......... .......... .......... .......... 74% 235M 0s 2019-12-09T09:43:04.071 INFO:teuthology.orchestra.run.smithi033.stderr: 55050K .......... .......... .......... .......... .......... 74% 262M 0s 2019-12-09T09:43:04.071 INFO:teuthology.orchestra.run.smithi033.stderr: 55100K .......... .......... .......... .......... .......... 74% 236M 0s 2019-12-09T09:43:04.071 INFO:teuthology.orchestra.run.smithi033.stderr: 55150K .......... .......... .......... .......... .......... 75% 209M 0s 2019-12-09T09:43:04.072 INFO:teuthology.orchestra.run.smithi033.stderr: 55200K .......... .......... .......... .......... .......... 75% 257M 0s 2019-12-09T09:43:04.072 INFO:teuthology.orchestra.run.smithi033.stderr: 55250K .......... .......... .......... .......... .......... 75% 226M 0s 2019-12-09T09:43:04.072 INFO:teuthology.orchestra.run.smithi033.stderr: 55300K .......... .......... .......... .......... .......... 75% 252M 0s 2019-12-09T09:43:04.072 INFO:teuthology.orchestra.run.smithi033.stderr: 55350K .......... .......... .......... .......... .......... 75% 229M 0s 2019-12-09T09:43:04.072 INFO:teuthology.orchestra.run.smithi033.stderr: 55400K .......... .......... .......... .......... .......... 75% 211M 0s 2019-12-09T09:43:04.073 INFO:teuthology.orchestra.run.smithi033.stderr: 55450K .......... .......... .......... .......... .......... 75% 264M 0s 2019-12-09T09:43:04.073 INFO:teuthology.orchestra.run.smithi033.stderr: 55500K .......... .......... .......... .......... .......... 75% 258M 0s 2019-12-09T09:43:04.073 INFO:teuthology.orchestra.run.smithi033.stderr: 55550K .......... .......... .......... .......... .......... 75% 221M 0s 2019-12-09T09:43:04.073 INFO:teuthology.orchestra.run.smithi033.stderr: 55600K .......... .......... .......... .......... .......... 75% 263M 0s 2019-12-09T09:43:04.074 INFO:teuthology.orchestra.run.smithi033.stderr: 55650K .......... .......... .......... .......... .......... 75% 247M 0s 2019-12-09T09:43:04.074 INFO:teuthology.orchestra.run.smithi033.stderr: 55700K .......... .......... .......... .......... .......... 75% 244M 0s 2019-12-09T09:43:04.075 INFO:teuthology.orchestra.run.smithi033.stderr: 55750K .......... .......... .......... .......... .......... 75% 208M 0s 2019-12-09T09:43:04.075 INFO:teuthology.orchestra.run.smithi033.stderr: 55800K .......... .......... .......... .......... .......... 75% 255M 0s 2019-12-09T09:43:04.075 INFO:teuthology.orchestra.run.smithi033.stderr: 55850K .......... .......... .......... .......... .......... 75% 262M 0s 2019-12-09T09:43:04.075 INFO:teuthology.orchestra.run.smithi033.stderr: 55900K .......... .......... .......... .......... .......... 76% 237M 0s 2019-12-09T09:43:04.076 INFO:teuthology.orchestra.run.smithi033.stderr: 55950K .......... .......... .......... .......... .......... 76% 208M 0s 2019-12-09T09:43:04.076 INFO:teuthology.orchestra.run.smithi033.stderr: 56000K .......... .......... .......... .......... .......... 76% 267M 0s 2019-12-09T09:43:04.076 INFO:teuthology.orchestra.run.smithi033.stderr: 56050K .......... .......... .......... .......... .......... 76% 238M 0s 2019-12-09T09:43:04.076 INFO:teuthology.orchestra.run.smithi033.stderr: 56100K .......... .......... .......... .......... .......... 76% 277M 0s 2019-12-09T09:43:04.076 INFO:teuthology.orchestra.run.smithi033.stderr: 56150K .......... .......... .......... .......... .......... 76% 222M 0s 2019-12-09T09:43:04.076 INFO:teuthology.orchestra.run.smithi033.stderr: 56200K .......... .......... .......... .......... .......... 76% 193M 0s 2019-12-09T09:43:04.077 INFO:teuthology.orchestra.run.smithi033.stderr: 56250K .......... .......... .......... .......... .......... 76% 216M 0s 2019-12-09T09:43:04.077 INFO:teuthology.orchestra.run.smithi033.stderr: 56300K .......... .......... .......... .......... .......... 76% 201M 0s 2019-12-09T09:43:04.077 INFO:teuthology.orchestra.run.smithi033.stderr: 56350K .......... .......... .......... .......... .......... 76% 194M 0s 2019-12-09T09:43:04.077 INFO:teuthology.orchestra.run.smithi033.stderr: 56400K .......... .......... .......... .......... .......... 76% 267M 0s 2019-12-09T09:43:04.077 INFO:teuthology.orchestra.run.smithi033.stderr: 56450K .......... .......... .......... .......... .......... 76% 260M 0s 2019-12-09T09:43:04.077 INFO:teuthology.orchestra.run.smithi033.stderr: 56500K .......... .......... .......... .......... .......... 76% 262M 0s 2019-12-09T09:43:04.077 INFO:teuthology.orchestra.run.smithi033.stderr: 56550K .......... .......... .......... .......... .......... 76% 215M 0s 2019-12-09T09:43:04.078 INFO:teuthology.orchestra.run.smithi033.stderr: 56600K .......... .......... .......... .......... .......... 77% 235M 0s 2019-12-09T09:43:04.078 INFO:teuthology.orchestra.run.smithi033.stderr: 56650K .......... .......... .......... .......... .......... 77% 263M 0s 2019-12-09T09:43:04.078 INFO:teuthology.orchestra.run.smithi033.stderr: 56700K .......... .......... .......... .......... .......... 77% 264M 0s 2019-12-09T09:43:04.078 INFO:teuthology.orchestra.run.smithi033.stderr: 56750K .......... .......... .......... .......... .......... 77% 220M 0s 2019-12-09T09:43:04.078 INFO:teuthology.orchestra.run.smithi033.stderr: 56800K .......... .......... .......... .......... .......... 77% 251M 0s 2019-12-09T09:43:04.078 INFO:teuthology.orchestra.run.smithi033.stderr: 56850K .......... .......... .......... .......... .......... 77% 257M 0s 2019-12-09T09:43:04.078 INFO:teuthology.orchestra.run.smithi033.stderr: 56900K .......... .......... .......... .......... .......... 77% 256M 0s 2019-12-09T09:43:04.079 INFO:teuthology.orchestra.run.smithi033.stderr: 56950K .......... .......... .......... .......... .......... 77% 211M 0s 2019-12-09T09:43:04.079 INFO:teuthology.orchestra.run.smithi033.stderr: 57000K .......... .......... .......... .......... .......... 77% 219M 0s 2019-12-09T09:43:04.079 INFO:teuthology.orchestra.run.smithi033.stderr: 57050K .......... .......... .......... .......... .......... 77% 253M 0s 2019-12-09T09:43:04.079 INFO:teuthology.orchestra.run.smithi033.stderr: 57100K .......... .......... .......... .......... .......... 77% 237M 0s 2019-12-09T09:43:04.079 INFO:teuthology.orchestra.run.smithi033.stderr: 57150K .......... .......... .......... .......... .......... 77% 216M 0s 2019-12-09T09:43:04.079 INFO:teuthology.orchestra.run.smithi033.stderr: 57200K .......... .......... .......... .......... .......... 77% 237M 0s 2019-12-09T09:43:04.079 INFO:teuthology.orchestra.run.smithi033.stderr: 57250K .......... .......... .......... .......... .......... 77% 232M 0s 2019-12-09T09:43:04.080 INFO:teuthology.orchestra.run.smithi033.stderr: 57300K .......... .......... .......... .......... .......... 77% 226M 0s 2019-12-09T09:43:04.080 INFO:teuthology.orchestra.run.smithi033.stderr: 57350K .......... .......... .......... .......... .......... 78% 217M 0s 2019-12-09T09:43:04.080 INFO:teuthology.orchestra.run.smithi033.stderr: 57400K .......... .......... .......... .......... .......... 78% 243M 0s 2019-12-09T09:43:04.080 INFO:teuthology.orchestra.run.smithi033.stderr: 57450K .......... .......... .......... .......... .......... 78% 222M 0s 2019-12-09T09:43:04.080 INFO:teuthology.orchestra.run.smithi033.stderr: 57500K .......... .......... .......... .......... .......... 78% 270M 0s 2019-12-09T09:43:04.080 INFO:teuthology.orchestra.run.smithi033.stderr: 57550K .......... .......... .......... .......... .......... 78% 220M 0s 2019-12-09T09:43:04.080 INFO:teuthology.orchestra.run.smithi033.stderr: 57600K .......... .......... .......... .......... .......... 78% 233M 0s 2019-12-09T09:43:04.081 INFO:teuthology.orchestra.run.smithi033.stderr: 57650K .......... .......... .......... .......... .......... 78% 235M 0s 2019-12-09T09:43:04.081 INFO:teuthology.orchestra.run.smithi033.stderr: 57700K .......... .......... .......... .......... .......... 78% 258M 0s 2019-12-09T09:43:04.081 INFO:teuthology.orchestra.run.smithi033.stderr: 57750K .......... .......... .......... .......... .......... 78% 218M 0s 2019-12-09T09:43:04.081 INFO:teuthology.orchestra.run.smithi033.stderr: 57800K .......... .......... .......... .......... .......... 78% 262M 0s 2019-12-09T09:43:04.081 INFO:teuthology.orchestra.run.smithi033.stderr: 57850K .......... .......... .......... .......... .......... 78% 274M 0s 2019-12-09T09:43:04.081 INFO:teuthology.orchestra.run.smithi033.stderr: 57900K .......... .......... .......... .......... .......... 78% 273M 0s 2019-12-09T09:43:04.081 INFO:teuthology.orchestra.run.smithi033.stderr: 57950K .......... .......... .......... .......... .......... 78% 237M 0s 2019-12-09T09:43:04.082 INFO:teuthology.orchestra.run.smithi033.stderr: 58000K .......... .......... .......... .......... .......... 78% 260M 0s 2019-12-09T09:43:04.082 INFO:teuthology.orchestra.run.smithi033.stderr: 58050K .......... .......... .......... .......... .......... 78% 248M 0s 2019-12-09T09:43:04.082 INFO:teuthology.orchestra.run.smithi033.stderr: 58100K .......... .......... .......... .......... .......... 79% 242M 0s 2019-12-09T09:43:04.082 INFO:teuthology.orchestra.run.smithi033.stderr: 58150K .......... .......... .......... .......... .......... 79% 222M 0s 2019-12-09T09:43:04.082 INFO:teuthology.orchestra.run.smithi033.stderr: 58200K .......... .......... .......... .......... .......... 79% 272M 0s 2019-12-09T09:43:04.082 INFO:teuthology.orchestra.run.smithi033.stderr: 58250K .......... .......... .......... .......... .......... 79% 264M 0s 2019-12-09T09:43:04.082 INFO:teuthology.orchestra.run.smithi033.stderr: 58300K .......... .......... .......... .......... .......... 79% 260M 0s 2019-12-09T09:43:04.083 INFO:teuthology.orchestra.run.smithi033.stderr: 58350K .......... .......... .......... .......... .......... 79% 217M 0s 2019-12-09T09:43:04.083 INFO:teuthology.orchestra.run.smithi033.stderr: 58400K .......... .......... .......... .......... .......... 79% 272M 0s 2019-12-09T09:43:04.083 INFO:teuthology.orchestra.run.smithi033.stderr: 58450K .......... .......... .......... .......... .......... 79% 254M 0s 2019-12-09T09:43:04.083 INFO:teuthology.orchestra.run.smithi033.stderr: 58500K .......... .......... .......... .......... .......... 79% 249M 0s 2019-12-09T09:43:04.083 INFO:teuthology.orchestra.run.smithi033.stderr: 58550K .......... .......... .......... .......... .......... 79% 215M 0s 2019-12-09T09:43:04.083 INFO:teuthology.orchestra.run.smithi033.stderr: 58600K .......... .......... .......... .......... .......... 79% 244M 0s 2019-12-09T09:43:04.083 INFO:teuthology.orchestra.run.smithi033.stderr: 58650K .......... .......... .......... .......... .......... 79% 239M 0s 2019-12-09T09:43:04.084 INFO:teuthology.orchestra.run.smithi033.stderr: 58700K .......... .......... .......... .......... .......... 79% 227M 0s 2019-12-09T09:43:04.084 INFO:teuthology.orchestra.run.smithi033.stderr: 58750K .......... .......... .......... .......... .......... 79% 205M 0s 2019-12-09T09:43:04.084 INFO:teuthology.orchestra.run.smithi033.stderr: 58800K .......... .......... .......... .......... .......... 80% 232M 0s 2019-12-09T09:43:04.084 INFO:teuthology.orchestra.run.smithi033.stderr: 58850K .......... .......... .......... .......... .......... 80% 240M 0s 2019-12-09T09:43:04.084 INFO:teuthology.orchestra.run.smithi033.stderr: 58900K .......... .......... .......... .......... .......... 80% 258M 0s 2019-12-09T09:43:04.084 INFO:teuthology.orchestra.run.smithi033.stderr: 58950K .......... .......... .......... .......... .......... 80% 276M 0s 2019-12-09T09:43:04.084 INFO:teuthology.orchestra.run.smithi033.stderr: 59000K .......... .......... .......... .......... .......... 80% 308M 0s 2019-12-09T09:43:04.085 INFO:teuthology.orchestra.run.smithi033.stderr: 59050K .......... .......... .......... .......... .......... 80% 237M 0s 2019-12-09T09:43:04.085 INFO:teuthology.orchestra.run.smithi033.stderr: 59100K .......... .......... .......... .......... .......... 80% 172M 0s 2019-12-09T09:43:04.085 INFO:teuthology.orchestra.run.smithi033.stderr: 59150K .......... .......... .......... .......... .......... 80% 193M 0s 2019-12-09T09:43:04.085 INFO:teuthology.orchestra.run.smithi033.stderr: 59200K .......... .......... .......... .......... .......... 80% 261M 0s 2019-12-09T09:43:04.085 INFO:teuthology.orchestra.run.smithi033.stderr: 59250K .......... .......... .......... .......... .......... 80% 262M 0s 2019-12-09T09:43:04.085 INFO:teuthology.orchestra.run.smithi033.stderr: 59300K .......... .......... .......... .......... .......... 80% 193M 0s 2019-12-09T09:43:04.085 INFO:teuthology.orchestra.run.smithi033.stderr: 59350K .......... .......... .......... .......... .......... 80% 223M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59400K .......... .......... .......... .......... .......... 80% 270M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59450K .......... .......... .......... .......... .......... 80% 257M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59500K .......... .......... .......... .......... .......... 80% 251M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59550K .......... .......... .......... .......... .......... 81% 219M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59600K .......... .......... .......... .......... .......... 81% 251M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59650K .......... .......... .......... .......... .......... 81% 262M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59700K .......... .......... .......... .......... .......... 81% 277M 0s 2019-12-09T09:43:04.086 INFO:teuthology.orchestra.run.smithi033.stderr: 59750K .......... .......... .......... .......... .......... 81% 235M 0s 2019-12-09T09:43:04.087 INFO:teuthology.orchestra.run.smithi033.stderr: 59800K .......... .......... .......... .......... .......... 81% 270M 0s 2019-12-09T09:43:04.087 INFO:teuthology.orchestra.run.smithi033.stderr: 59850K .......... .......... .......... .......... .......... 81% 267M 0s 2019-12-09T09:43:04.087 INFO:teuthology.orchestra.run.smithi033.stderr: 59900K .......... .......... .......... .......... .......... 81% 256M 0s 2019-12-09T09:43:04.087 INFO:teuthology.orchestra.run.smithi033.stderr: 59950K .......... .......... .......... .......... .......... 81% 210M 0s 2019-12-09T09:43:04.087 INFO:teuthology.orchestra.run.smithi033.stderr: 60000K .......... .......... .......... .......... .......... 81% 261M 0s 2019-12-09T09:43:04.087 INFO:teuthology.orchestra.run.smithi033.stderr: 60050K .......... .......... .......... .......... .......... 81% 259M 0s 2019-12-09T09:43:04.087 INFO:teuthology.orchestra.run.smithi033.stderr: 60100K .......... .......... .......... .......... .......... 81% 254M 0s 2019-12-09T09:43:04.088 INFO:teuthology.orchestra.run.smithi033.stderr: 60150K .......... .......... .......... .......... .......... 81% 193M 0s 2019-12-09T09:43:04.088 INFO:teuthology.orchestra.run.smithi033.stderr: 60200K .......... .......... .......... .......... .......... 81% 244M 0s 2019-12-09T09:43:04.088 INFO:teuthology.orchestra.run.smithi033.stderr: 60250K .......... .......... .......... .......... .......... 81% 241M 0s 2019-12-09T09:43:04.088 INFO:teuthology.orchestra.run.smithi033.stderr: 60300K .......... .......... .......... .......... .......... 82% 247M 0s 2019-12-09T09:43:04.088 INFO:teuthology.orchestra.run.smithi033.stderr: 60350K .......... .......... .......... .......... .......... 82% 181M 0s 2019-12-09T09:43:04.088 INFO:teuthology.orchestra.run.smithi033.stderr: 60400K .......... .......... .......... .......... .......... 82% 213M 0s 2019-12-09T09:43:04.088 INFO:teuthology.orchestra.run.smithi033.stderr: 60450K .......... .......... .......... .......... .......... 82% 246M 0s 2019-12-09T09:43:04.089 INFO:teuthology.orchestra.run.smithi033.stderr: 60500K .......... .......... .......... .......... .......... 82% 236M 0s 2019-12-09T09:43:04.089 INFO:teuthology.orchestra.run.smithi033.stderr: 60550K .......... .......... .......... .......... .......... 82% 224M 0s 2019-12-09T09:43:04.089 INFO:teuthology.orchestra.run.smithi033.stderr: 60600K .......... .......... .......... .......... .......... 82% 244M 0s 2019-12-09T09:43:04.089 INFO:teuthology.orchestra.run.smithi033.stderr: 60650K .......... .......... .......... .......... .......... 82% 235M 0s 2019-12-09T09:43:04.089 INFO:teuthology.orchestra.run.smithi033.stderr: 60700K .......... .......... .......... .......... .......... 82% 260M 0s 2019-12-09T09:43:04.089 INFO:teuthology.orchestra.run.smithi033.stderr: 60750K .......... .......... .......... .......... .......... 82% 225M 0s 2019-12-09T09:43:04.089 INFO:teuthology.orchestra.run.smithi033.stderr: 60800K .......... .......... .......... .......... .......... 82% 237M 0s 2019-12-09T09:43:04.090 INFO:teuthology.orchestra.run.smithi033.stderr: 60850K .......... .......... .......... .......... .......... 82% 272M 0s 2019-12-09T09:43:04.090 INFO:teuthology.orchestra.run.smithi033.stderr: 60900K .......... .......... .......... .......... .......... 82% 259M 0s 2019-12-09T09:43:04.090 INFO:teuthology.orchestra.run.smithi033.stderr: 60950K .......... .......... .......... .......... .......... 82% 205M 0s 2019-12-09T09:43:04.090 INFO:teuthology.orchestra.run.smithi033.stderr: 61000K .......... .......... .......... .......... .......... 82% 240M 0s 2019-12-09T09:43:04.090 INFO:teuthology.orchestra.run.smithi033.stderr: 61050K .......... .......... .......... .......... .......... 83% 267M 0s 2019-12-09T09:43:04.090 INFO:teuthology.orchestra.run.smithi033.stderr: 61100K .......... .......... .......... .......... .......... 83% 279M 0s 2019-12-09T09:43:04.090 INFO:teuthology.orchestra.run.smithi033.stderr: 61150K .......... .......... .......... .......... .......... 83% 199M 0s 2019-12-09T09:43:04.091 INFO:teuthology.orchestra.run.smithi033.stderr: 61200K .......... .......... .......... .......... .......... 83% 271M 0s 2019-12-09T09:43:04.091 INFO:teuthology.orchestra.run.smithi033.stderr: 61250K .......... .......... .......... .......... .......... 83% 263M 0s 2019-12-09T09:43:04.091 INFO:teuthology.orchestra.run.smithi033.stderr: 61300K .......... .......... .......... .......... .......... 83% 191M 0s 2019-12-09T09:43:04.091 INFO:teuthology.orchestra.run.smithi033.stderr: 61350K .......... .......... .......... .......... .......... 83% 193M 0s 2019-12-09T09:43:04.091 INFO:teuthology.orchestra.run.smithi033.stderr: 61400K .......... .......... .......... .......... .......... 83% 270M 0s 2019-12-09T09:43:04.091 INFO:teuthology.orchestra.run.smithi033.stderr: 61450K .......... .......... .......... .......... .......... 83% 242M 0s 2019-12-09T09:43:04.091 INFO:teuthology.orchestra.run.smithi033.stderr: 61500K .......... .......... .......... .......... .......... 83% 200M 0s 2019-12-09T09:43:04.092 INFO:teuthology.orchestra.run.smithi033.stderr: 61550K .......... .......... .......... .......... .......... 83% 226M 0s 2019-12-09T09:43:04.092 INFO:teuthology.orchestra.run.smithi033.stderr: 61600K .......... .......... .......... .......... .......... 83% 281M 0s 2019-12-09T09:43:04.092 INFO:teuthology.orchestra.run.smithi033.stderr: 61650K .......... .......... .......... .......... .......... 83% 231M 0s 2019-12-09T09:43:04.092 INFO:teuthology.orchestra.run.smithi033.stderr: 61700K .......... .......... .......... .......... .......... 83% 240M 0s 2019-12-09T09:43:04.092 INFO:teuthology.orchestra.run.smithi033.stderr: 61750K .......... .......... .......... .......... .......... 84% 204M 0s 2019-12-09T09:43:04.092 INFO:teuthology.orchestra.run.smithi033.stderr: 61800K .......... .......... .......... .......... .......... 84% 243M 0s 2019-12-09T09:43:04.092 INFO:teuthology.orchestra.run.smithi033.stderr: 61850K .......... .......... .......... .......... .......... 84% 243M 0s 2019-12-09T09:43:04.093 INFO:teuthology.orchestra.run.smithi033.stderr: 61900K .......... .......... .......... .......... .......... 84% 248M 0s 2019-12-09T09:43:04.093 INFO:teuthology.orchestra.run.smithi033.stderr: 61950K .......... .......... .......... .......... .......... 84% 206M 0s 2019-12-09T09:43:04.093 INFO:teuthology.orchestra.run.smithi033.stderr: 62000K .......... .......... .......... .......... .......... 84% 239M 0s 2019-12-09T09:43:04.093 INFO:teuthology.orchestra.run.smithi033.stderr: 62050K .......... .......... .......... .......... .......... 84% 249M 0s 2019-12-09T09:43:04.093 INFO:teuthology.orchestra.run.smithi033.stderr: 62100K .......... .......... .......... .......... .......... 84% 247M 0s 2019-12-09T09:43:04.093 INFO:teuthology.orchestra.run.smithi033.stderr: 62150K .......... .......... .......... .......... .......... 84% 204M 0s 2019-12-09T09:43:04.093 INFO:teuthology.orchestra.run.smithi033.stderr: 62200K .......... .......... .......... .......... .......... 84% 223M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62250K .......... .......... .......... .......... .......... 84% 254M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62300K .......... .......... .......... .......... .......... 84% 254M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62350K .......... .......... .......... .......... .......... 84% 225M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62400K .......... .......... .......... .......... .......... 84% 241M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62450K .......... .......... .......... .......... .......... 84% 237M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62500K .......... .......... .......... .......... .......... 85% 274M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62550K .......... .......... .......... .......... .......... 85% 220M 0s 2019-12-09T09:43:04.094 INFO:teuthology.orchestra.run.smithi033.stderr: 62600K .......... .......... .......... .......... .......... 85% 251M 0s 2019-12-09T09:43:04.095 INFO:teuthology.orchestra.run.smithi033.stderr: 62650K .......... .......... .......... .......... .......... 85% 277M 0s 2019-12-09T09:43:04.095 INFO:teuthology.orchestra.run.smithi033.stderr: 62700K .......... .......... .......... .......... .......... 85% 268M 0s 2019-12-09T09:43:04.095 INFO:teuthology.orchestra.run.smithi033.stderr: 62750K .......... .......... .......... .......... .......... 85% 220M 0s 2019-12-09T09:43:04.095 INFO:teuthology.orchestra.run.smithi033.stderr: 62800K .......... .......... .......... .......... .......... 85% 248M 0s 2019-12-09T09:43:04.095 INFO:teuthology.orchestra.run.smithi033.stderr: 62850K .......... .......... .......... .......... .......... 85% 266M 0s 2019-12-09T09:43:04.095 INFO:teuthology.orchestra.run.smithi033.stderr: 62900K .......... .......... .......... .......... .......... 85% 280M 0s 2019-12-09T09:43:04.096 INFO:teuthology.orchestra.run.smithi033.stderr: 62950K .......... .......... .......... .......... .......... 85% 221M 0s 2019-12-09T09:43:04.096 INFO:teuthology.orchestra.run.smithi033.stderr: 63000K .......... .......... .......... .......... .......... 85% 249M 0s 2019-12-09T09:43:04.096 INFO:teuthology.orchestra.run.smithi033.stderr: 63050K .......... .......... .......... .......... .......... 85% 231M 0s 2019-12-09T09:43:04.096 INFO:teuthology.orchestra.run.smithi033.stderr: 63100K .......... .......... .......... .......... .......... 85% 253M 0s 2019-12-09T09:43:04.097 INFO:teuthology.orchestra.run.smithi033.stderr: 63150K .......... .......... .......... .......... .......... 85% 198M 0s 2019-12-09T09:43:04.097 INFO:teuthology.orchestra.run.smithi033.stderr: 63200K .......... .......... .......... .......... .......... 85% 226M 0s 2019-12-09T09:43:04.097 INFO:teuthology.orchestra.run.smithi033.stderr: 63250K .......... .......... .......... .......... .......... 86% 239M 0s 2019-12-09T09:43:04.097 INFO:teuthology.orchestra.run.smithi033.stderr: 63300K .......... .......... .......... .......... .......... 86% 235M 0s 2019-12-09T09:43:04.097 INFO:teuthology.orchestra.run.smithi033.stderr: 63350K .......... .......... .......... .......... .......... 86% 169M 0s 2019-12-09T09:43:04.097 INFO:teuthology.orchestra.run.smithi033.stderr: 63400K .......... .......... .......... .......... .......... 86% 257M 0s 2019-12-09T09:43:04.097 INFO:teuthology.orchestra.run.smithi033.stderr: 63450K .......... .......... .......... .......... .......... 86% 250M 0s 2019-12-09T09:43:04.098 INFO:teuthology.orchestra.run.smithi033.stderr: 63500K .......... .......... .......... .......... .......... 86% 166M 0s 2019-12-09T09:43:04.098 INFO:teuthology.orchestra.run.smithi033.stderr: 63550K .......... .......... .......... .......... .......... 86% 210M 0s 2019-12-09T09:43:04.098 INFO:teuthology.orchestra.run.smithi033.stderr: 63600K .......... .......... .......... .......... .......... 86% 199M 0s 2019-12-09T09:43:04.098 INFO:teuthology.orchestra.run.smithi033.stderr: 63650K .......... .......... .......... .......... .......... 86% 229M 0s 2019-12-09T09:43:04.098 INFO:teuthology.orchestra.run.smithi033.stderr: 63700K .......... .......... .......... .......... .......... 86% 221M 0s 2019-12-09T09:43:04.098 INFO:teuthology.orchestra.run.smithi033.stderr: 63750K .......... .......... .......... .......... .......... 86% 216M 0s 2019-12-09T09:43:04.098 INFO:teuthology.orchestra.run.smithi033.stderr: 63800K .......... .......... .......... .......... .......... 86% 255M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 63850K .......... .......... .......... .......... .......... 86% 230M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 63900K .......... .......... .......... .......... .......... 86% 256M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 63950K .......... .......... .......... .......... .......... 87% 227M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 64000K .......... .......... .......... .......... .......... 87% 225M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 64050K .......... .......... .......... .......... .......... 87% 262M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 64100K .......... .......... .......... .......... .......... 87% 245M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 64150K .......... .......... .......... .......... .......... 87% 190M 0s 2019-12-09T09:43:04.099 INFO:teuthology.orchestra.run.smithi033.stderr: 64200K .......... .......... .......... .......... .......... 87% 261M 0s 2019-12-09T09:43:04.100 INFO:teuthology.orchestra.run.smithi033.stderr: 64250K .......... .......... .......... .......... .......... 87% 257M 0s 2019-12-09T09:43:04.100 INFO:teuthology.orchestra.run.smithi033.stderr: 64300K .......... .......... .......... .......... .......... 87% 271M 0s 2019-12-09T09:43:04.100 INFO:teuthology.orchestra.run.smithi033.stderr: 64350K .......... .......... .......... .......... .......... 87% 203M 0s 2019-12-09T09:43:04.100 INFO:teuthology.orchestra.run.smithi033.stderr: 64400K .......... .......... .......... .......... .......... 87% 250M 0s 2019-12-09T09:43:04.100 INFO:teuthology.orchestra.run.smithi033.stderr: 64450K .......... .......... .......... .......... .......... 87% 258M 0s 2019-12-09T09:43:04.100 INFO:teuthology.orchestra.run.smithi033.stderr: 64500K .......... .......... .......... .......... .......... 87% 272M 0s 2019-12-09T09:43:04.101 INFO:teuthology.orchestra.run.smithi033.stderr: 64550K .......... .......... .......... .......... .......... 87% 200M 0s 2019-12-09T09:43:04.101 INFO:teuthology.orchestra.run.smithi033.stderr: 64600K .......... .......... .......... .......... .......... 87% 256M 0s 2019-12-09T09:43:04.101 INFO:teuthology.orchestra.run.smithi033.stderr: 64650K .......... .......... .......... .......... .......... 87% 270M 0s 2019-12-09T09:43:04.101 INFO:teuthology.orchestra.run.smithi033.stderr: 64700K .......... .......... .......... .......... .......... 88% 277M 0s 2019-12-09T09:43:04.101 INFO:teuthology.orchestra.run.smithi033.stderr: 64750K .......... .......... .......... .......... .......... 88% 224M 0s 2019-12-09T09:43:04.101 INFO:teuthology.orchestra.run.smithi033.stderr: 64800K .......... .......... .......... .......... .......... 88% 244M 0s 2019-12-09T09:43:04.102 INFO:teuthology.orchestra.run.smithi033.stderr: 64850K .......... .......... .......... .......... .......... 88% 235M 0s 2019-12-09T09:43:04.102 INFO:teuthology.orchestra.run.smithi033.stderr: 64900K .......... .......... .......... .......... .......... 88% 248M 0s 2019-12-09T09:43:04.102 INFO:teuthology.orchestra.run.smithi033.stderr: 64950K .......... .......... .......... .......... .......... 88% 211M 0s 2019-12-09T09:43:04.102 INFO:teuthology.orchestra.run.smithi033.stderr: 65000K .......... .......... .......... .......... .......... 88% 256M 0s 2019-12-09T09:43:04.102 INFO:teuthology.orchestra.run.smithi033.stderr: 65050K .......... .......... .......... .......... .......... 88% 260M 0s 2019-12-09T09:43:04.102 INFO:teuthology.orchestra.run.smithi033.stderr: 65100K .......... .......... .......... .......... .......... 88% 242M 0s 2019-12-09T09:43:04.102 INFO:teuthology.orchestra.run.smithi033.stderr: 65150K .......... .......... .......... .......... .......... 88% 220M 0s 2019-12-09T09:43:04.103 INFO:teuthology.orchestra.run.smithi033.stderr: 65200K .......... .......... .......... .......... .......... 88% 277M 0s 2019-12-09T09:43:04.103 INFO:teuthology.orchestra.run.smithi033.stderr: 65250K .......... .......... .......... .......... .......... 88% 260M 0s 2019-12-09T09:43:04.103 INFO:teuthology.orchestra.run.smithi033.stderr: 65300K .......... .......... .......... .......... .......... 88% 241M 0s 2019-12-09T09:43:04.103 INFO:teuthology.orchestra.run.smithi033.stderr: 65350K .......... .......... .......... .......... .......... 88% 237M 0s 2019-12-09T09:43:04.103 INFO:teuthology.orchestra.run.smithi033.stderr: 65400K .......... .......... .......... .......... .......... 88% 241M 0s 2019-12-09T09:43:04.103 INFO:teuthology.orchestra.run.smithi033.stderr: 65450K .......... .......... .......... .......... .......... 89% 228M 0s 2019-12-09T09:43:04.103 INFO:teuthology.orchestra.run.smithi033.stderr: 65500K .......... .......... .......... .......... .......... 89% 247M 0s 2019-12-09T09:43:04.104 INFO:teuthology.orchestra.run.smithi033.stderr: 65550K .......... .......... .......... .......... .......... 89% 202M 0s 2019-12-09T09:43:04.104 INFO:teuthology.orchestra.run.smithi033.stderr: 65600K .......... .......... .......... .......... .......... 89% 218M 0s 2019-12-09T09:43:04.104 INFO:teuthology.orchestra.run.smithi033.stderr: 65650K .......... .......... .......... .......... .......... 89% 200M 0s 2019-12-09T09:43:04.104 INFO:teuthology.orchestra.run.smithi033.stderr: 65700K .......... .......... .......... .......... .......... 89% 223M 0s 2019-12-09T09:43:04.104 INFO:teuthology.orchestra.run.smithi033.stderr: 65750K .......... .......... .......... .......... .......... 89% 213M 0s 2019-12-09T09:43:04.104 INFO:teuthology.orchestra.run.smithi033.stderr: 65800K .......... .......... .......... .......... .......... 89% 238M 0s 2019-12-09T09:43:04.104 INFO:teuthology.orchestra.run.smithi033.stderr: 65850K .......... .......... .......... .......... .......... 89% 260M 0s 2019-12-09T09:43:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 65900K .......... .......... .......... .......... .......... 89% 233M 0s 2019-12-09T09:43:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 65950K .......... .......... .......... .......... .......... 89% 208M 0s 2019-12-09T09:43:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 66000K .......... .......... .......... .......... .......... 89% 234M 0s 2019-12-09T09:43:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 66050K .......... .......... .......... .......... .......... 89% 232M 0s 2019-12-09T09:43:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 66100K .......... .......... .......... .......... .......... 89% 288M 0s 2019-12-09T09:43:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 66150K .......... .......... .......... .......... .......... 89% 222M 0s 2019-12-09T09:43:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 66200K .......... .......... .......... .......... .......... 90% 242M 0s 2019-12-09T09:43:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 66250K .......... .......... .......... .......... .......... 90% 251M 0s 2019-12-09T09:43:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 66300K .......... .......... .......... .......... .......... 90% 264M 0s 2019-12-09T09:43:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 66350K .......... .......... .......... .......... .......... 90% 236M 0s 2019-12-09T09:43:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 66400K .......... .......... .......... .......... .......... 90% 252M 0s 2019-12-09T09:43:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 66450K .......... .......... .......... .......... .......... 90% 261M 0s 2019-12-09T09:43:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 66500K .......... .......... .......... .......... .......... 90% 266M 0s 2019-12-09T09:43:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 66550K .......... .......... .......... .......... .......... 90% 232M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66600K .......... .......... .......... .......... .......... 90% 263M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66650K .......... .......... .......... .......... .......... 90% 247M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66700K .......... .......... .......... .......... .......... 90% 248M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66750K .......... .......... .......... .......... .......... 90% 221M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66800K .......... .......... .......... .......... .......... 90% 239M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66850K .......... .......... .......... .......... .......... 90% 238M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66900K .......... .......... .......... .......... .......... 91% 221M 0s 2019-12-09T09:43:04.107 INFO:teuthology.orchestra.run.smithi033.stderr: 66950K .......... .......... .......... .......... .......... 91% 218M 0s 2019-12-09T09:43:04.108 INFO:teuthology.orchestra.run.smithi033.stderr: 67000K .......... .......... .......... .......... .......... 91% 243M 0s 2019-12-09T09:43:04.108 INFO:teuthology.orchestra.run.smithi033.stderr: 67050K .......... .......... .......... .......... .......... 91% 219M 0s 2019-12-09T09:43:04.108 INFO:teuthology.orchestra.run.smithi033.stderr: 67100K .......... .......... .......... .......... .......... 91% 192M 0s 2019-12-09T09:43:04.108 INFO:teuthology.orchestra.run.smithi033.stderr: 67150K .......... .......... .......... .......... .......... 91% 217M 0s 2019-12-09T09:43:04.108 INFO:teuthology.orchestra.run.smithi033.stderr: 67200K .......... .......... .......... .......... .......... 91% 224M 0s 2019-12-09T09:43:04.108 INFO:teuthology.orchestra.run.smithi033.stderr: 67250K .......... .......... .......... .......... .......... 91% 215M 0s 2019-12-09T09:43:04.108 INFO:teuthology.orchestra.run.smithi033.stderr: 67300K .......... .......... .......... .......... .......... 91% 242M 0s 2019-12-09T09:43:04.109 INFO:teuthology.orchestra.run.smithi033.stderr: 67350K .......... .......... .......... .......... .......... 91% 217M 0s 2019-12-09T09:43:04.109 INFO:teuthology.orchestra.run.smithi033.stderr: 67400K .......... .......... .......... .......... .......... 91% 279M 0s 2019-12-09T09:43:04.109 INFO:teuthology.orchestra.run.smithi033.stderr: 67450K .......... .......... .......... .......... .......... 91% 234M 0s 2019-12-09T09:43:04.109 INFO:teuthology.orchestra.run.smithi033.stderr: 67500K .......... .......... .......... .......... .......... 91% 228M 0s 2019-12-09T09:43:04.109 INFO:teuthology.orchestra.run.smithi033.stderr: 67550K .......... .......... .......... .......... .......... 91% 218M 0s 2019-12-09T09:43:04.109 INFO:teuthology.orchestra.run.smithi033.stderr: 67600K .......... .......... .......... .......... .......... 91% 244M 0s 2019-12-09T09:43:04.109 INFO:teuthology.orchestra.run.smithi033.stderr: 67650K .......... .......... .......... .......... .......... 92% 250M 0s 2019-12-09T09:43:04.110 INFO:teuthology.orchestra.run.smithi033.stderr: 67700K .......... .......... .......... .......... .......... 92% 264M 0s 2019-12-09T09:43:04.110 INFO:teuthology.orchestra.run.smithi033.stderr: 67750K .......... .......... .......... .......... .......... 92% 207M 0s 2019-12-09T09:43:04.110 INFO:teuthology.orchestra.run.smithi033.stderr: 67800K .......... .......... .......... .......... .......... 92% 266M 0s 2019-12-09T09:43:04.110 INFO:teuthology.orchestra.run.smithi033.stderr: 67850K .......... .......... .......... .......... .......... 92% 268M 0s 2019-12-09T09:43:04.110 INFO:teuthology.orchestra.run.smithi033.stderr: 67900K .......... .......... .......... .......... .......... 92% 256M 0s 2019-12-09T09:43:04.110 INFO:teuthology.orchestra.run.smithi033.stderr: 67950K .......... .......... .......... .......... .......... 92% 231M 0s 2019-12-09T09:43:04.110 INFO:teuthology.orchestra.run.smithi033.stderr: 68000K .......... .......... .......... .......... .......... 92% 249M 0s 2019-12-09T09:43:04.111 INFO:teuthology.orchestra.run.smithi033.stderr: 68050K .......... .......... .......... .......... .......... 92% 184M 0s 2019-12-09T09:43:04.111 INFO:teuthology.orchestra.run.smithi033.stderr: 68100K .......... .......... .......... .......... .......... 92% 248M 0s 2019-12-09T09:43:04.111 INFO:teuthology.orchestra.run.smithi033.stderr: 68150K .......... .......... .......... .......... .......... 92% 220M 0s 2019-12-09T09:43:04.111 INFO:teuthology.orchestra.run.smithi033.stderr: 68200K .......... .......... .......... .......... .......... 92% 194M 0s 2019-12-09T09:43:04.111 INFO:teuthology.orchestra.run.smithi033.stderr: 68250K .......... .......... .......... .......... .......... 92% 261M 0s 2019-12-09T09:43:04.111 INFO:teuthology.orchestra.run.smithi033.stderr: 68300K .......... .......... .......... .......... .......... 92% 260M 0s 2019-12-09T09:43:04.112 INFO:teuthology.orchestra.run.smithi033.stderr: 68350K .......... .......... .......... .......... .......... 92% 220M 0s 2019-12-09T09:43:04.112 INFO:teuthology.orchestra.run.smithi033.stderr: 68400K .......... .......... .......... .......... .......... 93% 259M 0s 2019-12-09T09:43:04.112 INFO:teuthology.orchestra.run.smithi033.stderr: 68450K .......... .......... .......... .......... .......... 93% 234M 0s 2019-12-09T09:43:04.112 INFO:teuthology.orchestra.run.smithi033.stderr: 68500K .......... .......... .......... .......... .......... 93% 219M 0s 2019-12-09T09:43:04.112 INFO:teuthology.orchestra.run.smithi033.stderr: 68550K .......... .......... .......... .......... .......... 93% 214M 0s 2019-12-09T09:43:04.112 INFO:teuthology.orchestra.run.smithi033.stderr: 68600K .......... .......... .......... .......... .......... 93% 249M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 68650K .......... .......... .......... .......... .......... 93% 227M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 68700K .......... .......... .......... .......... .......... 93% 250M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 68750K .......... .......... .......... .......... .......... 93% 235M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 68800K .......... .......... .......... .......... .......... 93% 243M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 68850K .......... .......... .......... .......... .......... 93% 256M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 68900K .......... .......... .......... .......... .......... 93% 229M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 68950K .......... .......... .......... .......... .......... 93% 208M 0s 2019-12-09T09:43:04.113 INFO:teuthology.orchestra.run.smithi033.stderr: 69000K .......... .......... .......... .......... .......... 93% 257M 0s 2019-12-09T09:43:04.114 INFO:teuthology.orchestra.run.smithi033.stderr: 69050K .......... .......... .......... .......... .......... 93% 241M 0s 2019-12-09T09:43:04.114 INFO:teuthology.orchestra.run.smithi033.stderr: 69100K .......... .......... .......... .......... .......... 94% 242M 0s 2019-12-09T09:43:04.114 INFO:teuthology.orchestra.run.smithi033.stderr: 69150K .......... .......... .......... .......... .......... 94% 206M 0s 2019-12-09T09:43:04.114 INFO:teuthology.orchestra.run.smithi033.stderr: 69200K .......... .......... .......... .......... .......... 94% 261M 0s 2019-12-09T09:43:04.114 INFO:teuthology.orchestra.run.smithi033.stderr: 69250K .......... .......... .......... .......... .......... 94% 265M 0s 2019-12-09T09:43:04.114 INFO:teuthology.orchestra.run.smithi033.stderr: 69300K .......... .......... .......... .......... .......... 94% 262M 0s 2019-12-09T09:43:04.114 INFO:teuthology.orchestra.run.smithi033.stderr: 69350K .......... .......... .......... .......... .......... 94% 217M 0s 2019-12-09T09:43:04.115 INFO:teuthology.orchestra.run.smithi033.stderr: 69400K .......... .......... .......... .......... .......... 94% 281M 0s 2019-12-09T09:43:04.115 INFO:teuthology.orchestra.run.smithi033.stderr: 69450K .......... .......... .......... .......... .......... 94% 268M 0s 2019-12-09T09:43:04.115 INFO:teuthology.orchestra.run.smithi033.stderr: 69500K .......... .......... .......... .......... .......... 94% 261M 0s 2019-12-09T09:43:04.115 INFO:teuthology.orchestra.run.smithi033.stderr: 69550K .......... .......... .......... .......... .......... 94% 179M 0s 2019-12-09T09:43:04.115 INFO:teuthology.orchestra.run.smithi033.stderr: 69600K .......... .......... .......... .......... .......... 94% 249M 0s 2019-12-09T09:43:04.115 INFO:teuthology.orchestra.run.smithi033.stderr: 69650K .......... .......... .......... .......... .......... 94% 231M 0s 2019-12-09T09:43:04.115 INFO:teuthology.orchestra.run.smithi033.stderr: 69700K .......... .......... .......... .......... .......... 94% 202M 0s 2019-12-09T09:43:04.116 INFO:teuthology.orchestra.run.smithi033.stderr: 69750K .......... .......... .......... .......... .......... 94% 219M 0s 2019-12-09T09:43:04.116 INFO:teuthology.orchestra.run.smithi033.stderr: 69800K .......... .......... .......... .......... .......... 94% 217M 0s 2019-12-09T09:43:04.116 INFO:teuthology.orchestra.run.smithi033.stderr: 69850K .......... .......... .......... .......... .......... 95% 220M 0s 2019-12-09T09:43:04.116 INFO:teuthology.orchestra.run.smithi033.stderr: 69900K .......... .......... .......... .......... .......... 95% 266M 0s 2019-12-09T09:43:04.116 INFO:teuthology.orchestra.run.smithi033.stderr: 69950K .......... .......... .......... .......... .......... 95% 226M 0s 2019-12-09T09:43:04.116 INFO:teuthology.orchestra.run.smithi033.stderr: 70000K .......... .......... .......... .......... .......... 95% 253M 0s 2019-12-09T09:43:04.116 INFO:teuthology.orchestra.run.smithi033.stderr: 70050K .......... .......... .......... .......... .......... 95% 233M 0s 2019-12-09T09:43:04.117 INFO:teuthology.orchestra.run.smithi033.stderr: 70100K .......... .......... .......... .......... .......... 95% 262M 0s 2019-12-09T09:43:04.117 INFO:teuthology.orchestra.run.smithi033.stderr: 70150K .......... .......... .......... .......... .......... 95% 233M 0s 2019-12-09T09:43:04.117 INFO:teuthology.orchestra.run.smithi033.stderr: 70200K .......... .......... .......... .......... .......... 95% 252M 0s 2019-12-09T09:43:04.117 INFO:teuthology.orchestra.run.smithi033.stderr: 70250K .......... .......... .......... .......... .......... 95% 218M 0s 2019-12-09T09:43:04.117 INFO:teuthology.orchestra.run.smithi033.stderr: 70300K .......... .......... .......... .......... .......... 95% 241M 0s 2019-12-09T09:43:04.117 INFO:teuthology.orchestra.run.smithi033.stderr: 70350K .......... .......... .......... .......... .......... 95% 230M 0s 2019-12-09T09:43:04.117 INFO:teuthology.orchestra.run.smithi033.stderr: 70400K .......... .......... .......... .......... .......... 95% 198M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70450K .......... .......... .......... .......... .......... 95% 241M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70500K .......... .......... .......... .......... .......... 95% 228M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70550K .......... .......... .......... .......... .......... 95% 201M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70600K .......... .......... .......... .......... .......... 96% 236M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70650K .......... .......... .......... .......... .......... 96% 232M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70700K .......... .......... .......... .......... .......... 96% 248M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70750K .......... .......... .......... .......... .......... 96% 218M 0s 2019-12-09T09:43:04.118 INFO:teuthology.orchestra.run.smithi033.stderr: 70800K .......... .......... .......... .......... .......... 96% 255M 0s 2019-12-09T09:43:04.119 INFO:teuthology.orchestra.run.smithi033.stderr: 70850K .......... .......... .......... .......... .......... 96% 247M 0s 2019-12-09T09:43:04.119 INFO:teuthology.orchestra.run.smithi033.stderr: 70900K .......... .......... .......... .......... .......... 96% 261M 0s 2019-12-09T09:43:04.119 INFO:teuthology.orchestra.run.smithi033.stderr: 70950K .......... .......... .......... .......... .......... 96% 208M 0s 2019-12-09T09:43:04.119 INFO:teuthology.orchestra.run.smithi033.stderr: 71000K .......... .......... .......... .......... .......... 96% 251M 0s 2019-12-09T09:43:04.119 INFO:teuthology.orchestra.run.smithi033.stderr: 71050K .......... .......... .......... .......... .......... 96% 271M 0s 2019-12-09T09:43:04.119 INFO:teuthology.orchestra.run.smithi033.stderr: 71100K .......... .......... .......... .......... .......... 96% 237M 0s 2019-12-09T09:43:04.119 INFO:teuthology.orchestra.run.smithi033.stderr: 71150K .......... .......... .......... .......... .......... 96% 227M 0s 2019-12-09T09:43:04.120 INFO:teuthology.orchestra.run.smithi033.stderr: 71200K .......... .......... .......... .......... .......... 96% 277M 0s 2019-12-09T09:43:04.120 INFO:teuthology.orchestra.run.smithi033.stderr: 71250K .......... .......... .......... .......... .......... 96% 256M 0s 2019-12-09T09:43:04.120 INFO:teuthology.orchestra.run.smithi033.stderr: 71300K .......... .......... .......... .......... .......... 96% 213M 0s 2019-12-09T09:43:04.120 INFO:teuthology.orchestra.run.smithi033.stderr: 71350K .......... .......... .......... .......... .......... 97% 204M 0s 2019-12-09T09:43:04.120 INFO:teuthology.orchestra.run.smithi033.stderr: 71400K .......... .......... .......... .......... .......... 97% 262M 0s 2019-12-09T09:43:04.120 INFO:teuthology.orchestra.run.smithi033.stderr: 71450K .......... .......... .......... .......... .......... 97% 257M 0s 2019-12-09T09:43:04.120 INFO:teuthology.orchestra.run.smithi033.stderr: 71500K .......... .......... .......... .......... .......... 97% 259M 0s 2019-12-09T09:43:04.121 INFO:teuthology.orchestra.run.smithi033.stderr: 71550K .......... .......... .......... .......... .......... 97% 223M 0s 2019-12-09T09:43:04.121 INFO:teuthology.orchestra.run.smithi033.stderr: 71600K .......... .......... .......... .......... .......... 97% 240M 0s 2019-12-09T09:43:04.121 INFO:teuthology.orchestra.run.smithi033.stderr: 71650K .......... .......... .......... .......... .......... 97% 245M 0s 2019-12-09T09:43:04.121 INFO:teuthology.orchestra.run.smithi033.stderr: 71700K .......... .......... .......... .......... .......... 97% 243M 0s 2019-12-09T09:43:04.121 INFO:teuthology.orchestra.run.smithi033.stderr: 71750K .......... .......... .......... .......... .......... 97% 245M 0s 2019-12-09T09:43:04.121 INFO:teuthology.orchestra.run.smithi033.stderr: 71800K .......... .......... .......... .......... .......... 97% 255M 0s 2019-12-09T09:43:04.121 INFO:teuthology.orchestra.run.smithi033.stderr: 71850K .......... .......... .......... .......... .......... 97% 228M 0s 2019-12-09T09:43:04.122 INFO:teuthology.orchestra.run.smithi033.stderr: 71900K .......... .......... .......... .......... .......... 97% 254M 0s 2019-12-09T09:43:04.122 INFO:teuthology.orchestra.run.smithi033.stderr: 71950K .......... .......... .......... .......... .......... 97% 232M 0s 2019-12-09T09:43:04.122 INFO:teuthology.orchestra.run.smithi033.stderr: 72000K .......... .......... .......... .......... .......... 97% 238M 0s 2019-12-09T09:43:04.122 INFO:teuthology.orchestra.run.smithi033.stderr: 72050K .......... .......... .......... .......... .......... 98% 249M 0s 2019-12-09T09:43:04.122 INFO:teuthology.orchestra.run.smithi033.stderr: 72100K .......... .......... .......... .......... .......... 98% 257M 0s 2019-12-09T09:43:04.122 INFO:teuthology.orchestra.run.smithi033.stderr: 72150K .......... .......... .......... .......... .......... 98% 226M 0s 2019-12-09T09:43:04.122 INFO:teuthology.orchestra.run.smithi033.stderr: 72200K .......... .......... .......... .......... .......... 98% 267M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72250K .......... .......... .......... .......... .......... 98% 278M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72300K .......... .......... .......... .......... .......... 98% 266M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72350K .......... .......... .......... .......... .......... 98% 222M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72400K .......... .......... .......... .......... .......... 98% 256M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72450K .......... .......... .......... .......... .......... 98% 273M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72500K .......... .......... .......... .......... .......... 98% 276M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72550K .......... .......... .......... .......... .......... 98% 295M 0s 2019-12-09T09:43:04.123 INFO:teuthology.orchestra.run.smithi033.stderr: 72600K .......... .......... .......... .......... .......... 98% 362M 0s 2019-12-09T09:43:04.124 INFO:teuthology.orchestra.run.smithi033.stderr: 72650K .......... .......... .......... .......... .......... 98% 292M 0s 2019-12-09T09:43:04.124 INFO:teuthology.orchestra.run.smithi033.stderr: 72700K .......... .......... .......... .......... .......... 98% 275M 0s 2019-12-09T09:43:04.124 INFO:teuthology.orchestra.run.smithi033.stderr: 72750K .......... .......... .......... .......... .......... 98% 231M 0s 2019-12-09T09:43:04.124 INFO:teuthology.orchestra.run.smithi033.stderr: 72800K .......... .......... .......... .......... .......... 99% 285M 0s 2019-12-09T09:43:04.124 INFO:teuthology.orchestra.run.smithi033.stderr: 72850K .......... .......... .......... .......... .......... 99% 274M 0s 2019-12-09T09:43:04.124 INFO:teuthology.orchestra.run.smithi033.stderr: 72900K .......... .......... .......... .......... .......... 99% 275M 0s 2019-12-09T09:43:04.124 INFO:teuthology.orchestra.run.smithi033.stderr: 72950K .......... .......... .......... .......... .......... 99% 236M 0s 2019-12-09T09:43:04.125 INFO:teuthology.orchestra.run.smithi033.stderr: 73000K .......... .......... .......... .......... .......... 99% 278M 0s 2019-12-09T09:43:04.125 INFO:teuthology.orchestra.run.smithi033.stderr: 73050K .......... .......... .......... .......... .......... 99% 280M 0s 2019-12-09T09:43:04.125 INFO:teuthology.orchestra.run.smithi033.stderr: 73100K .......... .......... .......... .......... .......... 99% 278M 0s 2019-12-09T09:43:04.125 INFO:teuthology.orchestra.run.smithi033.stderr: 73150K .......... .......... .......... .......... .......... 99% 233M 0s 2019-12-09T09:43:04.125 INFO:teuthology.orchestra.run.smithi033.stderr: 73200K .......... .......... .......... .......... .......... 99% 267M 0s 2019-12-09T09:43:04.125 INFO:teuthology.orchestra.run.smithi033.stderr: 73250K .......... .......... .......... .......... .......... 99% 277M 0s 2019-12-09T09:43:04.125 INFO:teuthology.orchestra.run.smithi033.stderr: 73300K .......... .......... .......... .......... .......... 99% 269M 0s 2019-12-09T09:43:04.126 INFO:teuthology.orchestra.run.smithi033.stderr: 73350K .......... .......... .......... .......... .......... 99% 245M 0s 2019-12-09T09:43:04.126 INFO:teuthology.orchestra.run.smithi033.stderr: 73400K .......... .......... .......... .......... .......... 99% 282M 0s 2019-12-09T09:43:04.126 INFO:teuthology.orchestra.run.smithi033.stderr: 73450K .......... .......... .......... .......... .......... 99% 242M 0s 2019-12-09T09:43:04.126 INFO:teuthology.orchestra.run.smithi033.stderr: 73500K .......... .......... .......... .......... .......... 99% 249M 0s 2019-12-09T09:43:04.126 INFO:teuthology.orchestra.run.smithi033.stderr: 73550K ........ 100% 298M=0.3s 2019-12-09T09:43:04.126 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T09:43:04.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09 09:43:02 (213 MB/s) - ‘gradle-4.7-bin.zip’ saved [75323957/75323957] 2019-12-09T09:43:04.127 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T09:43:04.127 INFO:teuthology.orchestra.run.smithi033.stdout:Archive: gradle-4.7-bin.zip 2019-12-09T09:43:04.127 INFO:teuthology.orchestra.run.smithi033.stdout: creating: /opt/gradle/gradle-4.7/ 2019-12-09T09:43:04.127 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/getting-started.html 2019-12-09T09:43:04.127 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/LICENSE 2019-12-09T09:43:04.127 INFO:teuthology.orchestra.run.smithi033.stdout: creating: /opt/gradle/gradle-4.7/media/ 2019-12-09T09:43:04.128 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-64x64.png 2019-12-09T09:43:04.128 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-48x48.png 2019-12-09T09:43:04.128 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle.icns 2019-12-09T09:43:04.128 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-24x24.png 2019-12-09T09:43:04.128 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-256x256.png 2019-12-09T09:43:04.128 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-512x512.png 2019-12-09T09:43:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-16x16.png 2019-12-09T09:43:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-32x32.png 2019-12-09T09:43:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-128x128.png 2019-12-09T09:43:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: creating: /opt/gradle/gradle-4.7/init.d/ 2019-12-09T09:43:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/init.d/readme.txt 2019-12-09T09:43:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/NOTICE 2019-12-09T09:43:04.129 INFO:teuthology.orchestra.run.smithi033.stdout: creating: /opt/gradle/gradle-4.7/bin/ 2019-12-09T09:43:04.130 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle.bat 2019-12-09T09:43:04.130 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle 2019-12-09T09:43:04.130 INFO:teuthology.orchestra.run.smithi033.stdout: creating: /opt/gradle/gradle-4.7/lib/ 2019-12-09T09:43:04.130 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/annotations-13.0.jar 2019-12-09T09:43:04.130 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-0.16.3.jar 2019-12-09T09:43:04.130 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-builders-0.16.3.jar 2019-12-09T09:43:04.131 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-models-0.16.3.jar 2019-12-09T09:43:04.131 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-compiler-embeddable-1.2.31.jar 2019-12-09T09:43:04.131 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-reflect-1.2.31.jar 2019-12-09T09:43:04.131 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-sam-with-receiver-compiler-plugin-1.2.31.jar 2019-12-09T09:43:04.131 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-script-runtime-1.2.31.jar 2019-12-09T09:43:04.131 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-1.2.31.jar 2019-12-09T09:43:04.132 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk7-1.2.31.jar 2019-12-09T09:43:04.132 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk8-1.2.31.jar 2019-12-09T09:43:04.132 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-launcher-4.7.jar 2019-12-09T09:43:04.132 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-runtime-api-info-4.7.jar 2019-12-09T09:43:04.132 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-tooling-api-4.7.jar 2019-12-09T09:43:04.132 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-wrapper-4.7.jar 2019-12-09T09:43:04.132 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-installation-beacon-4.7.jar 2019-12-09T09:43:04.133 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-4.7.jar 2019-12-09T09:43:04.133 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-jvm-services-4.7.jar 2019-12-09T09:43:04.133 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-api-4.7.jar 2019-12-09T09:43:04.133 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-cache-4.7.jar 2019-12-09T09:43:04.133 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-logging-4.7.jar 2019-12-09T09:43:04.133 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-process-services-4.7.jar 2019-12-09T09:43:04.133 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-resources-4.7.jar 2019-12-09T09:43:04.134 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-persistent-cache-4.7.jar 2019-12-09T09:43:04.134 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-native-4.7.jar 2019-12-09T09:43:04.134 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-groovy-4.7.jar 2019-12-09T09:43:04.134 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-core-4.7.jar 2019-12-09T09:43:04.134 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-groovy-4.7.jar 2019-12-09T09:43:04.134 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-messaging-4.7.jar 2019-12-09T09:43:04.134 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-4.7.jar 2019-12-09T09:43:04.135 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-option-4.7.jar 2019-12-09T09:43:04.135 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-cli-4.7.jar 2019-12-09T09:43:04.135 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-docs-4.7.jar 2019-12-09T09:43:04.135 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-commons-6.1.1.jar 2019-12-09T09:43:04.135 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-analysis-6.1.1.jar 2019-12-09T09:43:04.135 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-tree-6.1.1.jar 2019-12-09T09:43:04.136 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/kryo-2.20.jar 2019-12-09T09:43:04.136 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/reflectasm-1.07-shaded.jar 2019-12-09T09:43:04.136 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-6.1.1.jar 2019-12-09T09:43:04.136 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-io-2.4.jar 2019-12-09T09:43:04.136 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcl-over-slf4j-1.7.16.jar 2019-12-09T09:43:04.137 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/jul-to-slf4j-1.7.16.jar 2019-12-09T09:43:04.137 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/log4j-over-slf4j-1.7.16.jar 2019-12-09T09:43:04.137 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/slf4j-api-1.7.16.jar 2019-12-09T09:43:04.137 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/guava-jdk5-17.0.jar 2019-12-09T09:43:04.137 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/jsr305-2.0.1.jar 2019-12-09T09:43:04.137 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/fastutil-7.2.1-min.jar 2019-12-09T09:43:04.137 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-lang-2.6.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcip-annotations-1.0.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/groovy-all-2.4.12.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-1.9.9.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/javax.inject-1.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-collections-3.2.2.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-0.14.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-compress-1.14.jar 2019-12-09T09:43:04.138 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/xml-apis-1.4.01.jar 2019-12-09T09:43:04.139 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/jansi-1.14.jar 2019-12-09T09:43:04.139 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-launcher-1.9.9.jar 2019-12-09T09:43:04.139 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libcpp-0.14.jar 2019-12-09T09:43:04.139 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libstdcpp-0.14.jar 2019-12-09T09:43:04.139 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libcpp-0.14.jar 2019-12-09T09:43:04.139 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libstdcpp-0.14.jar 2019-12-09T09:43:04.139 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-0.14.jar 2019-12-09T09:43:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-0.14.jar 2019-12-09T09:43:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-amd64-0.14.jar 2019-12-09T09:43:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-i386-0.14.jar 2019-12-09T09:43:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-amd64-0.14.jar 2019-12-09T09:43:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-i386-0.14.jar 2019-12-09T09:43:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses5-0.14.jar 2019-12-09T09:43:04.140 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses6-0.14.jar 2019-12-09T09:43:04.141 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses5-0.14.jar 2019-12-09T09:43:04.141 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses6-0.14.jar 2019-12-09T09:43:04.141 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/minlog-1.2.jar 2019-12-09T09:43:04.141 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/objenesis-1.2.jar 2019-12-09T09:43:04.141 INFO:teuthology.orchestra.run.smithi033.stdout: creating: /opt/gradle/gradle-4.7/lib/plugins/ 2019-12-09T09:43:04.141 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-announce-4.7.jar 2019-12-09T09:43:04.141 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-antlr-4.7.jar 2019-12-09T09:43:04.142 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-code-quality-4.7.jar 2019-12-09T09:43:04.142 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-osgi-4.7.jar 2019-12-09T09:43:04.142 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-signing-4.7.jar 2019-12-09T09:43:04.142 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-native-4.7.jar 2019-12-09T09:43:04.142 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-native-4.7.jar 2019-12-09T09:43:04.142 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-native-4.7.jar 2019-12-09T09:43:04.142 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-development-4.7.jar 2019-12-09T09:43:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-maven-4.7.jar 2019-12-09T09:43:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-play-4.7.jar 2019-12-09T09:43:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-4.7.jar 2019-12-09T09:43:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-scala-4.7.jar 2019-12-09T09:43:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-comparison-4.7.jar 2019-12-09T09:43:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ear-4.7.jar 2019-12-09T09:43:04.143 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-play-4.7.jar 2019-12-09T09:43:04.144 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-javascript-4.7.jar 2019-12-09T09:43:04.144 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ivy-4.7.jar 2019-12-09T09:43:04.144 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-jacoco-4.7.jar 2019-12-09T09:43:04.144 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-init-4.7.jar 2019-12-09T09:43:04.144 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugins-4.7.jar 2019-12-09T09:43:04.144 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-tooling-api-builders-4.7.jar 2019-12-09T09:43:04.144 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-test-kit-4.7.jar 2019-12-09T09:43:04.145 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-composite-builds-4.7.jar 2019-12-09T09:43:04.145 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-groovy-4.7.jar 2019-12-09T09:43:04.145 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-scala-4.7.jar 2019-12-09T09:43:04.145 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-junit-platform-4.7.jar 2019-12-09T09:43:04.145 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-jvm-4.7.jar 2019-12-09T09:43:04.145 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-java-4.7.jar 2019-12-09T09:43:04.145 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-jvm-4.7.jar 2019-12-09T09:43:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-jvm-4.7.jar 2019-12-09T09:43:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-native-4.7.jar 2019-12-09T09:43:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-diagnostics-4.7.jar 2019-12-09T09:43:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-base-4.7.jar 2019-12-09T09:43:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-reporting-4.7.jar 2019-12-09T09:43:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-publish-4.7.jar 2019-12-09T09:43:04.146 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-base-4.7.jar 2019-12-09T09:43:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-use-4.7.jar 2019-12-09T09:43:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-s3-4.7.jar 2019-12-09T09:43:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-gcs-4.7.jar 2019-12-09T09:43:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-cache-http-4.7.jar 2019-12-09T09:43:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-dependency-management-4.7.jar 2019-12-09T09:43:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-http-4.7.jar 2019-12-09T09:43:04.147 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-sftp-4.7.jar 2019-12-09T09:43:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-workers-4.7.jar 2019-12-09T09:43:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-version-control-4.7.jar 2019-12-09T09:43:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-s3-1.11.267.jar 2019-12-09T09:43:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-kms-1.11.267.jar 2019-12-09T09:43:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-core-1.11.267.jar 2019-12-09T09:43:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/org.eclipse.jgit-4.5.3.201708160445-r.jar 2019-12-09T09:43:04.148 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-2.4.jar 2019-12-09T09:43:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-shared4-2.4.jar 2019-12-09T09:43:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-services-storage-v1-rev116-1.23.0.jar 2019-12-09T09:43:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-client-1.23.0.jar 2019-12-09T09:43:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-oauth-client-1.23.0.jar 2019-12-09T09:43:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-jackson2-1.23.0.jar 2019-12-09T09:43:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-1.23.0.jar 2019-12-09T09:43:04.149 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpclient-4.5.5.jar 2019-12-09T09:43:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-codec-1.10.jar 2019-12-09T09:43:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-container-default-1.5.5.jar 2019-12-09T09:43:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xbean-reflect-3.4.jar 2019-12-09T09:43:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/asm-util-6.1.1.jar 2019-12-09T09:43:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jatl-0.2.2.jar 2019-12-09T09:43:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-cli-1.2.jar 2019-12-09T09:43:04.150 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/biz.aQute.bndlib-3.4.0.jar 2019-12-09T09:43:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ivy-2.2.0.jar 2019-12-09T09:43:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-compat-3.0.4.jar 2019-12-09T09:43:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-core-3.0.4.jar 2019-12-09T09:43:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-common-0.8-20100325.jar 2019-12-09T09:43:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-groovy-0.8-20100325.jar 2019-12-09T09:43:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-file-2.4.jar 2019-12-09T09:43:04.151 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-connector-wagon-1.13.1.jar 2019-12-09T09:43:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcpg-jdk15on-1.58.jar 2019-12-09T09:43:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/rhino-1.7R3.jar 2019-12-09T09:43:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gson-2.7.jar 2019-12-09T09:43:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/simple-4.1.21.jar 2019-12-09T09:43:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-plugin-api-3.0.4.jar 2019-12-09T09:43:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/testng-6.3.1.jar 2019-12-09T09:43:04.152 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/snakeyaml-1.6.jar 2019-12-09T09:43:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/dd-plist-1.20.jar 2019-12-09T09:43:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-4.12.jar 2019-12-09T09:43:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bsh-2.0b6.jar 2019-12-09T09:43:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-launcher-1.0.3.jar 2019-12-09T09:43:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcifs-1.3.17.jar 2019-12-09T09:43:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/nekohtml-1.9.20.jar 2019-12-09T09:43:04.153 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xercesImpl-2.11.0.jar 2019-12-09T09:43:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jsch-0.1.54.jar 2019-12-09T09:43:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jmespath-java-1.11.267.jar 2019-12-09T09:43:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-databind-2.8.11.jar 2019-12-09T09:43:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-core-2.8.11.jar 2019-12-09T09:43:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-annotations-2.8.11.jar 2019-12-09T09:43:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/joda-time-2.8.2.jar 2019-12-09T09:43:04.154 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcprov-jdk15on-1.58.jar 2019-12-09T09:43:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-aether-provider-3.0.4.jar 2019-12-09T09:43:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-impl-1.13.1.jar 2019-12-09T09:43:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-spi-1.13.1.jar 2019-12-09T09:43:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-util-1.13.1.jar 2019-12-09T09:43:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-api-1.13.1.jar 2019-12-09T09:43:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-provider-api-2.4.jar 2019-12-09T09:43:04.155 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-classworlds-2.4.jar 2019-12-09T09:43:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-builder-3.0.4.jar 2019-12-09T09:43:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-3.0.4.jar 2019-12-09T09:43:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-builder-3.0.4.jar 2019-12-09T09:43:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-artifact-3.0.4.jar 2019-12-09T09:43:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-3.0.4.jar 2019-12-09T09:43:04.156 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-sec-dispatcher-1.3.jar 2019-12-09T09:43:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-repository-metadata-3.0.4.jar 2019-12-09T09:43:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-utils-3.0.8.jar 2019-12-09T09:43:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ion-java-1.0.2.jar 2019-12-09T09:43:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpcore-4.4.9.jar 2019-12-09T09:43:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/hamcrest-core-1.3.jar 2019-12-09T09:43:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-engine-1.0.3.jar 2019-12-09T09:43:04.157 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-commons-1.0.3.jar 2019-12-09T09:43:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/apiguardian-api-1.0.0.jar 2019-12-09T09:43:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcommander-1.47.jar 2019-12-09T09:43:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-interpolation-1.14.jar 2019-12-09T09:43:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-component-annotations-1.5.5.jar 2019-12-09T09:43:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/opentest4j-1.0.0.jar 2019-12-09T09:43:04.158 INFO:teuthology.orchestra.run.smithi033.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-cipher-1.7.jar 2019-12-09T09:43:04.158 INFO:teuthology.orchestra.run.smithi033.stdout:export PATH=/opt/gradle/gradle-4.7/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin 2019-12-09T09:43:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:04.852 INFO:teuthology.orchestra.run.smithi033.stdout:Welcome to Gradle 4.7! 2019-12-09T09:43:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:04.852 INFO:teuthology.orchestra.run.smithi033.stdout:Here are the highlights of this release: 2019-12-09T09:43:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: - Incremental annotation processing 2019-12-09T09:43:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: - JDK 10 support 2019-12-09T09:43:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: - Grouped non-interactive console logs 2019-12-09T09:43:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: - Failed tests are re-run first for quicker feedback 2019-12-09T09:43:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:04.854 INFO:teuthology.orchestra.run.smithi033.stdout:For more details see https://docs.gradle.org/4.7/release-notes.html 2019-12-09T09:43:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:05.198 INFO:teuthology.orchestra.run.smithi033.stdout:------------------------------------------------------------ 2019-12-09T09:43:05.198 INFO:teuthology.orchestra.run.smithi033.stdout:Gradle 4.7 2019-12-09T09:43:05.198 INFO:teuthology.orchestra.run.smithi033.stdout:------------------------------------------------------------ 2019-12-09T09:43:05.198 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:05.198 INFO:teuthology.orchestra.run.smithi033.stdout:Build time: 2018-04-18 09:09:12 UTC 2019-12-09T09:43:05.199 INFO:teuthology.orchestra.run.smithi033.stdout:Revision: b9a962bf70638332300e7f810689cb2febbd4a6c 2019-12-09T09:43:05.199 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:05.199 INFO:teuthology.orchestra.run.smithi033.stdout:Groovy: 2.4.12 2019-12-09T09:43:05.200 INFO:teuthology.orchestra.run.smithi033.stdout:Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 2019-12-09T09:43:05.200 INFO:teuthology.orchestra.run.smithi033.stdout:JVM: 1.8.0_232 (Oracle Corporation 25.232-b09) 2019-12-09T09:43:05.200 INFO:teuthology.orchestra.run.smithi033.stdout:OS: Linux 3.10.0-1062.9.1.el7.x86_64 amd64 2019-12-09T09:43:05.200 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:05.204 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2019-12-09T09:43:05.204 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T09:43:05.204 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi033.front.sepia.ceph.com 2019-12-09T09:43:05.204 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T09:43:05.205 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi186.front.sepia.ceph.com 2019-12-09T09:43:05.205 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2019-12-09T09:43:05.652 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 's3dir': 'folder', 'endpoint': None, 'display_name': None, 'region': 'mexico', 'port': None, 'access_secret': None, 'SSE': 'AES256', 'is_secure': None, 'email': None, 'dir': '../data'}} 2019-12-09T09:43:05.653 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2019-12-09T09:43:05.653 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'NUJYIBQUDFTZLQBQYEYA', 's3dir': 'folder', 'endpoint': 'smithi033.front.sepia.ceph.com:80', 'display_name': 'Ms. tester.client.0', 'user_id': 'tester.client.0', 'region': 'mexico', 'port': 80, 'access_secret': 'QiUwXk+8yWRvaZl5HKnjb527by0ixlsXGgI+gPGl3mK+1u9R7jFuKA==', 'host': 'smithi033.front.sepia.ceph.com', 'SSE': 'AES256', 'is_secure': False, 'email': 'tester.client.0_test@test.test', 'dir': '../data'} 2019-12-09T09:43:05.653 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2019-12-09T09:43:05.653 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'NUJYIBQUDFTZLQBQYEYA', '--secret', 'QiUwXk+8yWRvaZl5HKnjb527by0ixlsXGgI+gPGl3mK+1u9R7jFuKA==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2019-12-09T09:43:05.654 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key NUJYIBQUDFTZLQBQYEYA --secret QiUwXk+8yWRvaZl5HKnjb527by0ixlsXGgI+gPGl3mK+1u9R7jFuKA== --email tester.client.0_test@test.test --cluster ceph 2019-12-09T09:43:05.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.847+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:05.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.881+0000 7ff917d92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:05.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.881+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.889+0000 7ff917d92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:05.879 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.889+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:05.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.899+0000 7ff917d92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:05.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.899+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:05.913 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:05.924+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:06.114 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.125+0000 7ff917d92ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:43:06.114 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.125+0000 7ff917d92ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:43:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.125+0000 7ff917d92ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:43:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.125+0000 7ff917d92ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:43:06.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.125+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.138+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:06.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.138+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.151+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:43:06.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.151+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.155 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.166+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:06.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.166+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.179+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:43:06.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.179+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.176 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.187+0000 7ff917d92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:06.176 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.187+0000 7ff917d92ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:43:06.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.187+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.197+0000 7ff917d92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:06.186 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.197+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.209+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.209+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.224+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:43:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.224+0000 7ff917d92ec0 20 zonegroup default 2019-12-09T09:43:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.224+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.232+0000 7ff917d92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.232+0000 7ff917d92ec0 10 Cannot find current period zone using local zone 2019-12-09T09:43:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.232+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.239+0000 7ff917d92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.239+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.252+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:06.242 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.252+0000 7ff917d92ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:06.250 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.260+0000 7ff917d92ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:43:06.250 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.260+0000 7ff917d92ec0 20 zone default found 2019-12-09T09:43:06.250 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.260+0000 7ff917d92ec0 20 started sync module instance, tier type = 2019-12-09T09:43:06.250 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.260+0000 7ff917d92ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:43:06.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.682+0000 7ff917d92ec0 20 add_watcher() i=0 2019-12-09T09:43:06.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.682+0000 7ff917d92ec0 20 add_watcher() i=1 2019-12-09T09:43:06.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.682+0000 7ff917d92ec0 20 add_watcher() i=2 2019-12-09T09:43:06.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.682+0000 7ff917d92ec0 20 add_watcher() i=3 2019-12-09T09:43:06.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.682+0000 7ff917d92ec0 20 add_watcher() i=4 2019-12-09T09:43:06.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.682+0000 7ff917d92ec0 20 add_watcher() i=5 2019-12-09T09:43:06.675 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.682+0000 7ff917d92ec0 20 add_watcher() i=6 2019-12-09T09:43:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.723+0000 7ff917d92ec0 20 add_watcher() i=7 2019-12-09T09:43:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.723+0000 7ff917d92ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:43:06.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.724+0000 7ff8d57e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:43:06.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.736+0000 7ff904fbc700 0 -- >> v1:172.21.15.186:6789/0 conn(0x56343d68cd70 legacy=0x56343d691ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T09:43:06.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:06.905+0000 7ff9057bd700 0 -- 172.21.15.33:0/494226425 >> [v2:172.21.15.186:6810/10132,v1:172.21.15.186:6811/10132] conn(0x56343d6a2b20 msgr2=0x56343d6a4f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:43:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.411+0000 7ff8477fe700 20 reqs_thread_entry: start 2019-12-09T09:43:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.411+0000 7ff8457fa700 20 reqs_thread_entry: start 2019-12-09T09:43:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.412+0000 7ff917d92ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:43:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.412+0000 7ff8337fe700 20 reqs_thread_entry: start 2019-12-09T09:43:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.412+0000 7ff917d92ec0 20 get_system_obj_state: rctx=0x7ffd332eef68 obj=default.rgw.meta:users.uid:tester.client.0 state=0x56343d706670 s->prefetch_data=0 2019-12-09T09:43:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.412+0000 7ff917d92ec0 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2019-12-09T09:43:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.421+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2019-12-09T09:43:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.421+0000 7ff917d92ec0 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T09:43:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.421+0000 7ff917d92ec0 20 get_system_obj_state: rctx=0x7ffd332eef78 obj=default.rgw.meta:users.email:tester.client.0_test@test.test state=0x56343d708400 s->prefetch_data=0 2019-12-09T09:43:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.421+0000 7ff917d92ec0 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2019-12-09T09:43:08.422 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.433+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2019-12-09T09:43:08.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.433+0000 7ff917d92ec0 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T09:43:08.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.433+0000 7ff917d92ec0 20 get_system_obj_state: rctx=0x7ffd332eef78 obj=default.rgw.meta:users.keys:NUJYIBQUDFTZLQBQYEYA state=0x56343d708400 s->prefetch_data=0 2019-12-09T09:43:08.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.433+0000 7ff917d92ec0 10 cache get: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA : miss 2019-12-09T09:43:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.441+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA info.flags=0x0 2019-12-09T09:43:08.430 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.441+0000 7ff917d92ec0 10 adding default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA to cache LRU end 2019-12-09T09:43:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.441+0000 7ff917d92ec0 20 get_system_obj_state: rctx=0x7ffd332eeb88 obj=default.rgw.meta:users.keys:NUJYIBQUDFTZLQBQYEYA state=0x56343d708400 s->prefetch_data=0 2019-12-09T09:43:08.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.441+0000 7ff917d92ec0 10 cache get: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA : type miss (requested=0x6, cached=0x0) 2019-12-09T09:43:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.448+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA info.flags=0x0 2019-12-09T09:43:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.448+0000 7ff917d92ec0 10 moving default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA to cache LRU end 2019-12-09T09:43:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.448+0000 7ff917d92ec0 20 get_system_obj_state: rctx=0x7ffd332eed08 obj=default.rgw.meta:users.keys:NUJYIBQUDFTZLQBQYEYA state=0x56343d708400 s->prefetch_data=0 2019-12-09T09:43:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.448+0000 7ff917d92ec0 10 cache get: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA : type miss (requested=0x6, cached=0x0) 2019-12-09T09:43:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.456+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA info.flags=0x0 2019-12-09T09:43:08.446 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.456+0000 7ff917d92ec0 10 moving default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA to cache LRU end 2019-12-09T09:43:08.505 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.516+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T09:43:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.516+0000 7ff917d92ec0 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T09:43:08.506 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.516+0000 7ff917d92ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=632 2019-12-09T09:43:08.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.528+0000 7ff8ea7fc700 10 RGWWatcher::handle_notify() notify_id 579820584968 cookie 94782368487056 notifier 4755 bl.length()=632 2019-12-09T09:43:08.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.528+0000 7ff8ea7fc700 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T09:43:08.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.528+0000 7ff8ea7fc700 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T09:43:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.806+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T09:43:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.806+0000 7ff917d92ec0 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T09:43:08.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.806+0000 7ff917d92ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=200 2019-12-09T09:43:08.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.818+0000 7ff8ea7fc700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 94782368481168 notifier 4755 bl.length()=200 2019-12-09T09:43:08.808 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.818+0000 7ff8ea7fc700 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T09:43:08.808 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:08.818+0000 7ff8ea7fc700 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T09:43:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.018+0000 7ff917d92ec0 10 cache put: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA info.flags=0x7 2019-12-09T09:43:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.018+0000 7ff917d92ec0 10 moving default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA to cache LRU end 2019-12-09T09:43:09.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.018+0000 7ff917d92ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=189 2019-12-09T09:43:09.016 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.027+0000 7ff8ea7fc700 10 RGWWatcher::handle_notify() notify_id 579820584963 cookie 94782368481168 notifier 4755 bl.length()=189 2019-12-09T09:43:09.016 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.027+0000 7ff8ea7fc700 10 cache put: name=default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA info.flags=0x7 2019-12-09T09:43:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.027+0000 7ff8ea7fc700 10 moving default.rgw.meta+users.keys+NUJYIBQUDFTZLQBQYEYA to cache LRU end 2019-12-09T09:43:09.157 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T09:43:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "tester.client.0", 2019-12-09T09:43:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Ms. tester.client.0", 2019-12-09T09:43:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "tester.client.0_test@test.test", 2019-12-09T09:43:09.157 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T09:43:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T09:43:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [], 2019-12-09T09:43:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [ 2019-12-09T09:43:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T09:43:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "tester.client.0", 2019-12-09T09:43:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "access_key": "NUJYIBQUDFTZLQBQYEYA", 2019-12-09T09:43:09.158 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "QiUwXk+8yWRvaZl5HKnjb527by0ixlsXGgI+gPGl3mK+1u9R7jFuKA==" 2019-12-09T09:43:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T09:43:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T09:43:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [], 2019-12-09T09:43:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T09:43:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T09:43:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T09:43:09.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T09:43:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T09:43:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T09:43:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T09:43:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T09:43:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T09:43:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T09:43:09.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T09:43:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T09:43:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T09:43:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T09:43:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T09:43:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T09:43:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T09:43:09.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T09:43:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T09:43:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T09:43:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T09:43:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T09:43:09.162 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T09:43:09.162 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:09.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.212+0000 7ff917d92ec0 20 remove_watcher() i=0 2019-12-09T09:43:09.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.212+0000 7ff917d92ec0 2 removed watcher, disabling cache 2019-12-09T09:43:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.290+0000 7ff917d92ec0 20 remove_watcher() i=1 2019-12-09T09:43:09.334 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.344+0000 7ff917d92ec0 20 remove_watcher() i=2 2019-12-09T09:43:09.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.382+0000 7ff917d92ec0 20 remove_watcher() i=3 2019-12-09T09:43:09.418 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.429+0000 7ff917d92ec0 20 remove_watcher() i=4 2019-12-09T09:43:09.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.468+0000 7ff917d92ec0 20 remove_watcher() i=5 2019-12-09T09:43:09.506 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.517+0000 7ff917d92ec0 20 remove_watcher() i=6 2019-12-09T09:43:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.563+0000 7ff917d92ec0 20 remove_watcher() i=7 2019-12-09T09:43:09.563 INFO:teuthology.orchestra.run.smithi033:> cat > /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2019-12-09T09:43:09.620 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2019-12-09T09:43:09.621 INFO:teuthology.orchestra.run.smithi033:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2019-12-09T09:43:09.692 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T09:43:09.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.857+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.892+0000 7f0c92c31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.892+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.898+0000 7f0c92c31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.898+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.906+0000 7f0c92c31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:09.896 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.906+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:09.919 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:09.930+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.100+0000 7f0c92c31ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:43:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.100+0000 7f0c92c31ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:43:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.100+0000 7f0c92c31ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:43:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.100+0000 7f0c92c31ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:43:10.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.100+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.109 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.120+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:10.110 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.120+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.128+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:43:10.118 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.128+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.139+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:10.129 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.139+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.148 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.158+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:43:10.148 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.159+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.166+0000 7f0c92c31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.166+0000 7f0c92c31ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:43:10.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.166+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.163 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.173+0000 7f0c92c31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:10.163 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.173+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.182+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:10.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.182+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.199+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:43:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.199+0000 7f0c92c31ec0 20 zonegroup default 2019-12-09T09:43:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.199+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.205+0000 7f0c92c31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.205+0000 7f0c92c31ec0 10 Cannot find current period zone using local zone 2019-12-09T09:43:10.194 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.205+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.210+0000 7f0c92c31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:10.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.210+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.218+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:10.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.218+0000 7f0c92c31ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.226+0000 7f0c92c31ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:43:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.226+0000 7f0c92c31ec0 20 zone default found 2019-12-09T09:43:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.226+0000 7f0c92c31ec0 20 started sync module instance, tier type = 2019-12-09T09:43:10.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.226+0000 7f0c92c31ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:43:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.444+0000 7f0c7f65a700 0 -- 172.21.15.33:0/3713657810 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f0c60035a50 msgr2=0x7f0c60037f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:43:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.541+0000 7f0c92c31ec0 20 add_watcher() i=0 2019-12-09T09:43:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.541+0000 7f0c92c31ec0 20 add_watcher() i=1 2019-12-09T09:43:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.541+0000 7f0c92c31ec0 20 add_watcher() i=2 2019-12-09T09:43:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.541+0000 7f0c92c31ec0 20 add_watcher() i=3 2019-12-09T09:43:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.541+0000 7f0c92c31ec0 20 add_watcher() i=4 2019-12-09T09:43:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.541+0000 7f0c92c31ec0 20 add_watcher() i=5 2019-12-09T09:43:10.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.541+0000 7f0c92c31ec0 20 add_watcher() i=6 2019-12-09T09:43:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.586+0000 7f0c92c31ec0 20 add_watcher() i=7 2019-12-09T09:43:10.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.586+0000 7f0c92c31ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:43:10.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.586+0000 7f0bd3fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:43:10.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:10.609+0000 7f0c7f65a700 0 -- 172.21.15.33:0/3697280485 >> v1:172.21.15.186:6789/0 conn(0x55ec8a163e60 legacy=0x55ec8a164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:43:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.171+0000 7f0bbe7fc700 20 reqs_thread_entry: start 2019-12-09T09:43:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.171+0000 7f0bb7fff700 20 reqs_thread_entry: start 2019-12-09T09:43:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.171+0000 7f0c92c31ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:43:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.172+0000 7f0bb67fc700 20 reqs_thread_entry: start 2019-12-09T09:43:12.162 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.172+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:43:12.219 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.230+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:12.240 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.250+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:12.251 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.262+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.324+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:43:12.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.381+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:12.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.397+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:12.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.433+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:12.423 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.433+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_rNNl0DHW1__qZTUjFwE5dBaPhDJpZcv', time=2019-12-09T11:42:42.864315+0000, chain.objs.size()=1 2019-12-09T09:43:12.424 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.433+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.cPkZmAdNhv9dLnxLKt5QclfnTTpXOYd_0 2019-12-09T09:43:12.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.487+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:12.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.611+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:43:12.649 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.660+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:12.689 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.699+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.734+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.734+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_UNq2FOcaTDLxIiZZGjFHLi9SnuC1jWP', time=2019-12-09T11:42:37.118296+0000, chain.objs.size()=2 2019-12-09T09:43:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.734+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.TebNpmYR0YR80oz75eIpYGat0h-hjkR_0 2019-12-09T09:43:12.724 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.734+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.TebNpmYR0YR80oz75eIpYGat0h-hjkR_1 2019-12-09T09:43:12.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:12.899+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.043+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:43:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.093+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:13.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.111+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.138+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.138+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.210', time=2019-12-09T11:42:44.757346+0000, chain.objs.size()=3 2019-12-09T09:43:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.138+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-STANDARD.2~cmcrYqU1cnx966que6fdXBLS3RbURxi.1 2019-12-09T09:43:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.138+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-STANDARD.2~cmcrYqU1cnx966que6fdXBLS3RbURxi.2 2019-12-09T09:43:13.129 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.138+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-STANDARD.2~cmcrYqU1cnx966que6fdXBLS3RbURxi.3 2019-12-09T09:43:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.264+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:13.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.388+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:43:13.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.453+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.482+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:43:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.40', time=2019-12-09T11:42:31.030926+0000, chain.objs.size()=4 2019-12-09T09:43:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3__multipart_foo.2~kPbGUV_loy90iXQIzGVn711q-jrZ7iX.1 2019-12-09T09:43:13.529 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3__multipart_foo.2~kPbGUV_loy90iXQIzGVn711q-jrZ7iX.2 2019-12-09T09:43:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3__multipart_foo.2~kPbGUV_loy90iXQIzGVn711q-jrZ7iX.3 2019-12-09T09:43:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.3__multipart_foo.2~kPbGUV_loy90iXQIzGVn711q-jrZ7iX.4 2019-12-09T09:43:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.112', time=2019-12-09T11:42:37.773159+0000, chain.objs.size()=2 2019-12-09T09:43:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.0hnQF7y-nYscDcgySxZvJlEalCbDU8Z_0 2019-12-09T09:43:13.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.539+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.0hnQF7y-nYscDcgySxZvJlEalCbDU8Z_1 2019-12-09T09:43:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.674+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:13.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.812+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:43:13.879 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.889+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:13.912 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.922+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:13.927 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.938+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:13.927 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.938+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_JICTi0rqsIxW3iMTh9vkigKt4VZhaHJ', time=2019-12-09T11:42:43.057991+0000, chain.objs.size()=1 2019-12-09T09:43:13.928 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.938+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.x3dhzAWIuto47aQle84dxhygmnmvpae_0 2019-12-09T09:43:13.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:13.979+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:14.121 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.131+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:43:14.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.210+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:14.228 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.238+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:14.251 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.262+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:14.251 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.262+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.242', time=2019-12-09T11:42:42.675871+0000, chain.objs.size()=3 2019-12-09T09:43:14.251 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.262+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-STANDARD.2~CbqB2NE5UAciOIXmyfxb5gjBwXe0eB3.1 2019-12-09T09:43:14.252 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.262+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-STANDARD.2~CbqB2NE5UAciOIXmyfxb5gjBwXe0eB3.2 2019-12-09T09:43:14.252 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.262+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-STANDARD.2~CbqB2NE5UAciOIXmyfxb5gjBwXe0eB3.3 2019-12-09T09:43:14.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.453+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:14.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.563+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:43:14.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.610+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:14.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.630+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.640+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:14.664 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.675+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:43:14.716 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.727+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:14.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.745+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.758+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.758+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.84', time=2019-12-09T11:42:46.646907+0000, chain.objs.size()=3 2019-12-09T09:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.758+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6__shadow_.BkNZXhfrDiZueIRF1JNDm2C9ZEebhZ-_0 2019-12-09T09:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.758+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6__shadow_.BkNZXhfrDiZueIRF1JNDm2C9ZEebhZ-_1 2019-12-09T09:43:14.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.758+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6__shadow_.BkNZXhfrDiZueIRF1JNDm2C9ZEebhZ-_2 2019-12-09T09:43:14.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:14.892+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.007+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:43:15.079 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.089+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:15.125 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.135+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:15.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.152+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:15.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.152+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.109', time=2019-12-09T11:42:36.903626+0000, chain.objs.size()=2 2019-12-09T09:43:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.152+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.lrs3pIbGSe_FO2gEy02zIY95ngUA8ZN_0 2019-12-09T09:43:15.143 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.152+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.lrs3pIbGSe_FO2gEy02zIY95ngUA8ZN_1 2019-12-09T09:43:15.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.208+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.293+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:43:15.348 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.359+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:15.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.378+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.428+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:15.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.428+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_hyK_gu_ODRFjlIXdRixGOxDzNHgdjXm', time=2019-12-09T11:42:37.970292+0000, chain.objs.size()=2 2019-12-09T09:43:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.428+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.ZnEsaePR_iD6BFguUsczQO37nmNS088_0 2019-12-09T09:43:15.418 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.428+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.ZnEsaePR_iD6BFguUsczQO37nmNS088_1 2019-12-09T09:43:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.566+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:15.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.661+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:43:15.732 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.743+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:15.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.771+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:15.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.789+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:15.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.845+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:43:15.899 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.910+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:15.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.936+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:15.939 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.949+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:15.988 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:15.998+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:43:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.053+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.076+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.090+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:16.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.090+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.203', time=2019-12-09T11:42:44.606735+0000, chain.objs.size()=5 2019-12-09T09:43:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.090+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-LUKEWARM.2~ttWATFuXIUOLkFCg22kUDM0HB6HPE4u.1 2019-12-09T09:43:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.090+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__shadow_foo-LUKEWARM.2~ttWATFuXIUOLkFCg22kUDM0HB6HPE4u.1_1 2019-12-09T09:43:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.090+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-LUKEWARM.2~ttWATFuXIUOLkFCg22kUDM0HB6HPE4u.2 2019-12-09T09:43:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.090+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__shadow_foo-LUKEWARM.2~ttWATFuXIUOLkFCg22kUDM0HB6HPE4u.2_1 2019-12-09T09:43:16.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.091+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-LUKEWARM.2~ttWATFuXIUOLkFCg22kUDM0HB6HPE4u.3 2019-12-09T09:43:16.263 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.274+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:16.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.358+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:43:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.400+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:16.405 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.416+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:16.413 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.424+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:16.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.458+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:43:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.493+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:16.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.508+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:16.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:43:16.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_QvBbFiVuaUosXs1TuQ9-Ha1MCG_iJ34', time=2019-12-09T11:42:38.580383+0000, chain.objs.size()=1 2019-12-09T09:43:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.C3EHCOhQ_4Cc4_Rh2yolnT_vTFWjwlg_1 2019-12-09T09:43:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.225', time=2019-12-09T11:42:40.800883+0000, chain.objs.size()=5 2019-12-09T09:43:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.1 2019-12-09T09:43:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.1_1 2019-12-09T09:43:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.2 2019-12-09T09:43:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.2_1 2019-12-09T09:43:16.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.523+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.3 2019-12-09T09:43:16.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.637+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:16.723 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.733+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:43:16.763 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.774+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:16.778 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.789+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:43:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.55', time=2019-12-09T11:42:32.782846+0000, chain.objs.size()=4 2019-12-09T09:43:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4__multipart_foo.2~fEAPuVt8_HT7afI3MAHaTyzUp8W3LJY.1 2019-12-09T09:43:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4__multipart_foo.2~fEAPuVt8_HT7afI3MAHaTyzUp8W3LJY.2 2019-12-09T09:43:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4__multipart_foo.2~fEAPuVt8_HT7afI3MAHaTyzUp8W3LJY.3 2019-12-09T09:43:16.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.4__multipart_foo.2~fEAPuVt8_HT7afI3MAHaTyzUp8W3LJY.4 2019-12-09T09:43:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.233', time=2019-12-09T11:42:41.657938+0000, chain.objs.size()=5 2019-12-09T09:43:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.1 2019-12-09T09:43:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.1_1 2019-12-09T09:43:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.2 2019-12-09T09:43:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.2_1 2019-12-09T09:43:16.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.3 2019-12-09T09:43:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.235', time=2019-12-09T11:42:42.138260+0000, chain.objs.size()=5 2019-12-09T09:43:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.1 2019-12-09T09:43:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.1_1 2019-12-09T09:43:16.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:16.809+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.2 2019-12-09T09:43:16.997 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.007+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.2_1 2019-12-09T09:43:17.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.025+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-LUKEWARM.2~p7iAJKMCgYbpBBW6BGvFgT3A6ntKWDV.3 2019-12-09T09:43:17.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.177+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.289+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:43:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.343+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:17.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.360+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T09:43:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_A5km80k2Kt0ho-8q_rLwMu7kftpb5xi', time=2019-12-09T11:42:37.430236+0000, chain.objs.size()=1 2019-12-09T09:43:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.H3nCkplYLiNWAOl6ARvSfuHv812FeZe_1 2019-12-09T09:43:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.245', time=2019-12-09T11:42:43.217184+0000, chain.objs.size()=3 2019-12-09T09:43:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-STANDARD.2~CbqB2NE5UAciOIXmyfxb5gjBwXe0eB3.1 2019-12-09T09:43:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-STANDARD.2~CbqB2NE5UAciOIXmyfxb5gjBwXe0eB3.2 2019-12-09T09:43:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-STANDARD.2~CbqB2NE5UAciOIXmyfxb5gjBwXe0eB3.3 2019-12-09T09:43:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.196', time=2019-12-09T11:42:44.404253+0000, chain.objs.size()=5 2019-12-09T09:43:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-FROZEN.2~gS8NOWQf5jkrQFJLfsX_rKQgGStOKu3.1 2019-12-09T09:43:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__shadow_foo-FROZEN.2~gS8NOWQf5jkrQFJLfsX_rKQgGStOKu3.1_1 2019-12-09T09:43:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-FROZEN.2~gS8NOWQf5jkrQFJLfsX_rKQgGStOKu3.2 2019-12-09T09:43:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__shadow_foo-FROZEN.2~gS8NOWQf5jkrQFJLfsX_rKQgGStOKu3.2_1 2019-12-09T09:43:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.380+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.8__multipart_foo-FROZEN.2~gS8NOWQf5jkrQFJLfsX_rKQgGStOKu3.3 2019-12-09T09:43:17.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.654+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:17.758 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.769+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:43:17.808 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.819+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:17.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.837+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.853+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.853+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_5YyNuvRzLNYPh9d7rbiUS7fa8vpJYBr', time=2019-12-09T11:42:38.949238+0000, chain.objs.size()=1 2019-12-09T09:43:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.853+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.fuF3BX9qC-_Na_9JGWjqi0bY3QbsCKb_0 2019-12-09T09:43:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.896+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:17.993 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:17.999+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:43:18.066 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.077+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.100+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:18.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.117+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.155 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.166+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:43:18.208 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.218+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.225 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.236+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:18.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.247+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.277 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.287+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:43:18.354 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.365+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.409+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.427+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.427+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_5kkz9bPGQBSaFgRoxedGxT1h9_JDSa4', time=2019-12-09T11:42:39.126993+0000, chain.objs.size()=1 2019-12-09T09:43:18.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.427+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.wHs8eOcMwkYtLue8fmLalJT9DA8L2-7_0 2019-12-09T09:43:18.468 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.479+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:18.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.604+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:43:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.656+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.666 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.676+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:18.677 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.687+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.737+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:43:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.806+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:18.835 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.843+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:18.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:43:18.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_ydrmbnPmPMLK1dglQ5asiIJd28-eoUT', time=2019-12-09T11:42:41.427790+0000, chain.objs.size()=2 2019-12-09T09:43:18.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.D7UbYHYkBFRQgL1zSweVE4xZD6C7vF1_1 2019-12-09T09:43:18.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.D7UbYHYkBFRQgL1zSweVE4xZD6C7vF1_2 2019-12-09T09:43:18.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_ruB1BfhKILjAQn_d_fPjx3ws4koTHKa', time=2019-12-09T11:42:41.861593+0000, chain.objs.size()=3 2019-12-09T09:43:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.i_gyN48-qMzUGsIbK0a74XSnj1eQDmK_0 2019-12-09T09:43:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.i_gyN48-qMzUGsIbK0a74XSnj1eQDmK_1 2019-12-09T09:43:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:18.865+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.i_gyN48-qMzUGsIbK0a74XSnj1eQDmK_2 2019-12-09T09:43:19.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.091+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.171+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:43:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.217+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.233+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:19.237 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:43:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_Po1ZWhqBuImCoqqbzw0hj3mC8fx4rIx', time=2019-12-09T11:42:42.457416+0000, chain.objs.size()=2 2019-12-09T09:43:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.YwpBvb_Vfq06PN3_D7CrGS2Lqtdc4lD_1 2019-12-09T09:43:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.YwpBvb_Vfq06PN3_D7CrGS2Lqtdc4lD_2 2019-12-09T09:43:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.85', time=2019-12-09T11:42:46.845871+0000, chain.objs.size()=3 2019-12-09T09:43:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6__shadow_.WYS6azBh_Fwa-9Aq-L1SLdTYaYI5PI6_0 2019-12-09T09:43:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6__shadow_.WYS6azBh_Fwa-9Aq-L1SLdTYaYI5PI6_1 2019-12-09T09:43:19.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.248+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.6__shadow_.WYS6azBh_Fwa-9Aq-L1SLdTYaYI5PI6_2 2019-12-09T09:43:19.476 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.487+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:19.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.572+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:43:19.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.636+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.660+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.677+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.678+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.108', time=2019-12-09T11:42:36.644279+0000, chain.objs.size()=2 2019-12-09T09:43:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.678+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.lrs3pIbGSe_FO2gEy02zIY95ngUA8ZN_0 2019-12-09T09:43:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.678+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.lrs3pIbGSe_FO2gEy02zIY95ngUA8ZN_1 2019-12-09T09:43:19.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.837+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:19.948 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:19.959+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:43:20.006 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.017+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:20.024 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.035+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.069+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.069+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='_8AaBRyOcBy5-yvRhi4b80LcnCFPSdg3', time=2019-12-09T11:42:40.997355+0000, chain.objs.size()=3 2019-12-09T09:43:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.069+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.e_myEV2QK0Yk8Y2WY4cByVwlb3-VZtQ_0 2019-12-09T09:43:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.069+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.e_myEV2QK0Yk8Y2WY4cByVwlb3-VZtQ_1 2019-12-09T09:43:20.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.069+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_.e_myEV2QK0Yk8Y2WY4cByVwlb3-VZtQ_2 2019-12-09T09:43:20.229 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.240+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:20.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.406+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:43:20.460 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.471+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:20.481 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.492+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:20.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.502+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:20.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.560+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:43:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.635+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:20.653 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.664+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.688+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.688+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.71', time=2019-12-09T11:42:34.617739+0000, chain.objs.size()=4 2019-12-09T09:43:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.688+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5__multipart_foo.2~MA4BIZpXTXj4ukZ90CF_Ja1jak5pUmP.1 2019-12-09T09:43:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.688+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5__multipart_foo.2~MA4BIZpXTXj4ukZ90CF_Ja1jak5pUmP.2 2019-12-09T09:43:20.677 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.688+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5__multipart_foo.2~MA4BIZpXTXj4ukZ90CF_Ja1jak5pUmP.3 2019-12-09T09:43:20.678 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.688+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.5__multipart_foo.2~MA4BIZpXTXj4ukZ90CF_Ja1jak5pUmP.4 2019-12-09T09:43:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.830+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:20.931 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.942+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:43:20.976 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:20.987+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:21.003 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.004+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:21.011 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.020+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T09:43:21.011 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.020+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.20', time=2019-12-09T11:42:29.373836+0000, chain.objs.size()=4 2019-12-09T09:43:21.011 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.020+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2__multipart_foo.2~miHatCh8vRy3X0JSCBHkiTaWB26Pil_.1 2019-12-09T09:43:21.011 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.020+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2__multipart_foo.2~miHatCh8vRy3X0JSCBHkiTaWB26Pil_.2 2019-12-09T09:43:21.012 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.020+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2__multipart_foo.2~miHatCh8vRy3X0JSCBHkiTaWB26Pil_.3 2019-12-09T09:43:21.012 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.020+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.2__multipart_foo.2~miHatCh8vRy3X0JSCBHkiTaWB26Pil_.4 2019-12-09T09:43:21.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.171+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:21.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.264+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:43:21.299 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.310+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:21.316 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.327+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:21.325 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.336+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:21.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.382+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:43:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.445+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:21.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.473+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:21.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T09:43:21.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.114', time=2019-12-09T11:42:38.238769+0000, chain.objs.size()=2 2019-12-09T09:43:21.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.0hnQF7y-nYscDcgySxZvJlEalCbDU8Z_0 2019-12-09T09:43:21.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.7__shadow_.0hnQF7y-nYscDcgySxZvJlEalCbDU8Z_1 2019-12-09T09:43:21.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.224', time=2019-12-09T11:42:40.615529+0000, chain.objs.size()=5 2019-12-09T09:43:21.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.1 2019-12-09T09:43:21.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.1_1 2019-12-09T09:43:21.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.2 2019-12-09T09:43:21.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__shadow_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.2_1 2019-12-09T09:43:21.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.491+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.9__multipart_foo-FROZEN.2~ASBRy9ZM7Luauhz4fVs3swWvXMoJH1B.3 2019-12-09T09:43:21.671 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.682+0000 7f0c92c31ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T09:43:21.853 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.857+0000 7f0c92c31ec0 20 remove_watcher() i=0 2019-12-09T09:43:21.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.857+0000 7f0c92c31ec0 2 removed watcher, disabling cache 2019-12-09T09:43:21.940 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:21.950+0000 7f0c92c31ec0 20 remove_watcher() i=1 2019-12-09T09:43:22.007 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.018+0000 7f0c92c31ec0 20 remove_watcher() i=2 2019-12-09T09:43:22.082 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.092+0000 7f0c92c31ec0 20 remove_watcher() i=3 2019-12-09T09:43:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.143+0000 7f0c92c31ec0 20 remove_watcher() i=4 2019-12-09T09:43:22.190 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.201+0000 7f0c92c31ec0 20 remove_watcher() i=5 2019-12-09T09:43:22.234 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.245+0000 7f0c92c31ec0 20 remove_watcher() i=6 2019-12-09T09:43:22.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.298+0000 7f0c92c31ec0 20 remove_watcher() i=7 2019-12-09T09:43:22.296 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T09:43:22.436 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.447+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.481 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.492+0000 7f6029fe7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:22.481 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.492+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.498+0000 7f6029fe7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:22.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.498+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.507+0000 7f6029fe7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:22.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.507+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.524+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.685+0000 7f6029fe7ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T09:43:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.685+0000 7f6029fe7ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T09:43:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.685+0000 7f6029fe7ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T09:43:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.685+0000 7f6029fe7ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T09:43:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.685+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.695+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.695+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.709+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:43:22.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.709+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.707 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.718+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:22.707 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.718+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.717 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.728+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:43:22.718 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.728+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.734+0000 7f6029fe7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.734+0000 7f6029fe7ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T09:43:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.735+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.742+0000 7f6029fe7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:22.732 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.742+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.759+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:22.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.759+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.775+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T09:43:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.775+0000 7f6029fe7ec0 20 zonegroup default 2019-12-09T09:43:22.765 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.775+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.785+0000 7f6029fe7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.785+0000 7f6029fe7ec0 10 Cannot find current period zone using local zone 2019-12-09T09:43:22.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.785+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.794+0000 7f6029fe7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T09:43:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.794+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.794 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.804+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T09:43:22.794 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.804+0000 7f6029fe7ec0 20 rados->read ofs=0 len=0 2019-12-09T09:43:22.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.820+0000 7f6029fe7ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T09:43:22.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.820+0000 7f6029fe7ec0 20 zone default found 2019-12-09T09:43:22.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.820+0000 7f6029fe7ec0 20 started sync module instance, tier type = 2019-12-09T09:43:22.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:22.820+0000 7f6029fe7ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T09:43:23.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.162+0000 7f6029fe7ec0 20 add_watcher() i=0 2019-12-09T09:43:23.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.162+0000 7f6029fe7ec0 20 add_watcher() i=1 2019-12-09T09:43:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.162+0000 7f6029fe7ec0 20 add_watcher() i=2 2019-12-09T09:43:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.162+0000 7f6029fe7ec0 20 add_watcher() i=3 2019-12-09T09:43:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.162+0000 7f6029fe7ec0 20 add_watcher() i=4 2019-12-09T09:43:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.162+0000 7f6029fe7ec0 20 add_watcher() i=5 2019-12-09T09:43:23.152 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.162+0000 7f6029fe7ec0 20 add_watcher() i=6 2019-12-09T09:43:23.187 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.198+0000 7f6029fe7ec0 20 add_watcher() i=7 2019-12-09T09:43:23.187 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.198+0000 7f6029fe7ec0 2 all 8 watchers are set, enabling cache 2019-12-09T09:43:23.187 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.198+0000 7f5fe7fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T09:43:23.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.204+0000 7f6017a12700 0 -- 172.21.15.33:0/217325662 >> v1:172.21.15.33:6789/0 conn(0x55e6aa19bf20 legacy=0x55e6aa1b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:43:23.230 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:23.240+0000 7f6016a10700 0 -- 172.21.15.33:0/2745094806 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x55e6aa1d0840 msgr2=0x55e6aa1d2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T09:43:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.087+0000 7f5f567fc700 20 reqs_thread_entry: start 2019-12-09T09:43:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.088+0000 7f5f4ffff700 20 reqs_thread_entry: start 2019-12-09T09:43:25.077 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.088+0000 7f6029fe7ec0 20 init_complete bucket index max shards: 0 2019-12-09T09:43:25.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.089+0000 7f5f4e7fc700 20 reqs_thread_entry: start 2019-12-09T09:43:25.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.089+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T09:43:25.148 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.158+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.188+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:25.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.199+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.259+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T09:43:25.315 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.325+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.355+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:25.359 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.370+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.410 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.421+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T09:43:25.489 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.500+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.541+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:25.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.563+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.597+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T09:43:25.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.635+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.655+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:25.652 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.663+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.702 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.712+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T09:43:25.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.770+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.792+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:25.791 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.802+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.830 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.841+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T09:43:25.879 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.890+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.905+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.915+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:25.956+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T09:43:26.009 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.019+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.034 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.045+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.058+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.101+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T09:43:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.163+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.179+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.189+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.208 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.219+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T09:43:26.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.262+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.277+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.286+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.317+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T09:43:26.389 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.385+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.408+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.410 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.420+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.462 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.471+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T09:43:26.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.522+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.533 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.543+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.544 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.553+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.594+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T09:43:26.647 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.658+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.666 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.677+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.685 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.686+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.719 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.730+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T09:43:26.794 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.805+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.814 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.825+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.827 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.835+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.876 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.887+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T09:43:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.949+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.981+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:26.984 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:26.994+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.044 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.055+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T09:43:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.115+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.148+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:27.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.171+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.226+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T09:43:27.270 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.281+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.297 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.297+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:27.297 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.306+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.397+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T09:43:27.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.444+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.489+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:27.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.498+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.534 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.545+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T09:43:27.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.623+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.655+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.670+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.722+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T09:43:27.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.787+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.807+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.819+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.861 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.871+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T09:43:27.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.947+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:27.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.975+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:27.978 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:27.988+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.022 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.032+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T09:43:28.077 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.087+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.111+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.120+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.167+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T09:43:28.205 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.216+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.235+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:28.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.249+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.286+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T09:43:28.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.378+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.398 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.409+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:28.413 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.424+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.470+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T09:43:28.534 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.545+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.575+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:28.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.592+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.632 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.643+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T09:43:28.681 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.692+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.697 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.709+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:28.708 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.719+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.755+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T09:43:28.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.812+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.834+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:28.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.843+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.879+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T09:43:28.946 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.957+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:28.986 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:28.997+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:28.997 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.008+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.055 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.066+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T09:43:29.110 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.120+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.130 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.140+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.153+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.185 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.196+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T09:43:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.247+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.273+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.282+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.315 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.326+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T09:43:29.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.381+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.391 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.402+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.411+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.444+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T09:43:29.481 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.492+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.502 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.513+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:29.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.522+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.559+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T09:43:29.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.640+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.663+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T09:43:29.664 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.675+0000 7f6029fe7ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T09:43:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.754+0000 7f6029fe7ec0 20 remove_watcher() i=0 2019-12-09T09:43:29.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.754+0000 7f6029fe7ec0 2 removed watcher, disabling cache 2019-12-09T09:43:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.801+0000 7f6029fe7ec0 20 remove_watcher() i=1 2019-12-09T09:43:29.847 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.858+0000 7f6029fe7ec0 20 remove_watcher() i=2 2019-12-09T09:43:29.903 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.914+0000 7f6029fe7ec0 20 remove_watcher() i=3 2019-12-09T09:43:29.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:29.972+0000 7f6029fe7ec0 20 remove_watcher() i=4 2019-12-09T09:43:30.035 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:30.045+0000 7f6029fe7ec0 20 remove_watcher() i=5 2019-12-09T09:43:30.097 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:30.108+0000 7f6029fe7ec0 20 remove_watcher() i=6 2019-12-09T09:43:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T09:43:30.165+0000 7f6029fe7ec0 20 remove_watcher() i=7 2019-12-09T09:43:30.164 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2019-12-09T09:43:30.767 INFO:teuthology.orchestra.run.smithi033.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2019-12-09T09:43:34.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.pom 2019-12-09T09:43:34.866 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.jar 2019-12-09T09:43:36.067 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :clean UP-TO-DATE 2019-12-09T09:43:36.166 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-bom/1.11.549/aws-java-sdk-bom-1.11.549.pom 2019-12-09T09:43:36.166 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.549/aws-java-sdk-pom-1.11.549.pom 2019-12-09T09:43:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.pom 2019-12-09T09:43:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.pom 2019-12-09T09:43:36.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.pom 2019-12-09T09:43:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.pom 2019-12-09T09:43:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.pom 2019-12-09T09:43:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.pom 2019-12-09T09:43:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.pom 2019-12-09T09:43:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom 2019-12-09T09:43:36.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-parent/2.44.0/selenium-parent-2.44.0.pom 2019-12-09T09:43:36.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.pom 2019-12-09T09:43:36.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.pom 2019-12-09T09:43:36.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.pom 2019-12-09T09:43:36.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.pom 2019-12-09T09:43:36.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom 2019-12-09T09:43:36.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.pom 2019-12-09T09:43:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.2/jackson-parent-2.6.2.pom 2019-12-09T09:43:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.5/httpcomponents-client-4.5.5.pom 2019-12-09T09:43:36.469 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom 2019-12-09T09:43:36.566 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/24/oss-parent-24.pom 2019-12-09T09:43:36.566 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/httpcomponents-parent-10.pom 2019-12-09T09:43:36.566 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2019-12-09T09:43:36.566 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2019-12-09T09:43:36.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.pom 2019-12-09T09:43:36.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.pom 2019-12-09T09:43:36.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.pom 2019-12-09T09:43:36.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.pom 2019-12-09T09:43:36.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.pom 2019-12-09T09:43:36.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.pom 2019-12-09T09:43:36.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.pom 2019-12-09T09:43:36.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.pom 2019-12-09T09:43:36.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.pom 2019-12-09T09:43:36.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.pom 2019-12-09T09:43:36.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.pom 2019-12-09T09:43:36.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.pom 2019-12-09T09:43:36.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.pom 2019-12-09T09:43:36.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.pom 2019-12-09T09:43:36.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.pom 2019-12-09T09:43:36.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.pom 2019-12-09T09:43:36.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.pom 2019-12-09T09:43:36.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.pom 2019-12-09T09:43:36.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.pom 2019-12-09T09:43:36.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.pom 2019-12-09T09:43:36.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.pom 2019-12-09T09:43:36.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.pom 2019-12-09T09:43:36.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.pom 2019-12-09T09:43:36.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.pom 2019-12-09T09:43:36.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.pom 2019-12-09T09:43:36.766 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.pom 2019-12-09T09:43:36.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.pom 2019-12-09T09:43:36.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.pom 2019-12-09T09:43:36.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.pom 2019-12-09T09:43:36.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.pom 2019-12-09T09:43:36.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.pom 2019-12-09T09:43:36.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.pom 2019-12-09T09:43:36.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.pom 2019-12-09T09:43:36.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.pom 2019-12-09T09:43:36.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.pom 2019-12-09T09:43:36.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.pom 2019-12-09T09:43:36.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.pom 2019-12-09T09:43:36.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.pom 2019-12-09T09:43:36.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.pom 2019-12-09T09:43:36.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.pom 2019-12-09T09:43:36.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.pom 2019-12-09T09:43:36.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.pom 2019-12-09T09:43:36.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.pom 2019-12-09T09:43:36.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.pom 2019-12-09T09:43:36.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.pom 2019-12-09T09:43:36.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.pom 2019-12-09T09:43:36.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.pom 2019-12-09T09:43:36.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.pom 2019-12-09T09:43:36.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.pom 2019-12-09T09:43:36.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.pom 2019-12-09T09:43:36.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.pom 2019-12-09T09:43:36.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.pom 2019-12-09T09:43:36.876 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.pom 2019-12-09T09:43:36.877 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.pom 2019-12-09T09:43:36.877 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.pom 2019-12-09T09:43:36.877 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.pom 2019-12-09T09:43:36.878 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.pom 2019-12-09T09:43:36.878 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.pom 2019-12-09T09:43:36.878 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.pom 2019-12-09T09:43:36.878 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.pom 2019-12-09T09:43:36.878 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.pom 2019-12-09T09:43:36.878 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.pom 2019-12-09T09:43:36.879 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.pom 2019-12-09T09:43:36.879 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.pom 2019-12-09T09:43:36.879 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.pom 2019-12-09T09:43:36.880 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.pom 2019-12-09T09:43:36.880 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.pom 2019-12-09T09:43:36.881 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.pom 2019-12-09T09:43:36.881 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.pom 2019-12-09T09:43:36.881 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.pom 2019-12-09T09:43:36.883 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.pom 2019-12-09T09:43:36.883 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.pom 2019-12-09T09:43:36.883 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.pom 2019-12-09T09:43:36.883 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.pom 2019-12-09T09:43:36.884 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.pom 2019-12-09T09:43:36.884 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.pom 2019-12-09T09:43:36.884 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.pom 2019-12-09T09:43:36.884 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.pom 2019-12-09T09:43:36.884 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.pom 2019-12-09T09:43:36.885 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.pom 2019-12-09T09:43:36.885 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.pom 2019-12-09T09:43:36.885 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.pom 2019-12-09T09:43:36.968 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.pom 2019-12-09T09:43:36.968 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.pom 2019-12-09T09:43:36.968 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.pom 2019-12-09T09:43:36.968 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.pom 2019-12-09T09:43:36.969 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.pom 2019-12-09T09:43:36.969 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.pom 2019-12-09T09:43:36.969 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.pom 2019-12-09T09:43:36.969 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.pom 2019-12-09T09:43:36.969 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.pom 2019-12-09T09:43:36.969 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.pom 2019-12-09T09:43:36.970 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.pom 2019-12-09T09:43:36.970 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.pom 2019-12-09T09:43:36.970 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.pom 2019-12-09T09:43:36.970 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.pom 2019-12-09T09:43:36.970 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.pom 2019-12-09T09:43:36.970 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.pom 2019-12-09T09:43:36.971 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.pom 2019-12-09T09:43:36.971 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.pom 2019-12-09T09:43:36.971 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.pom 2019-12-09T09:43:36.971 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.pom 2019-12-09T09:43:36.971 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.pom 2019-12-09T09:43:36.972 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.pom 2019-12-09T09:43:36.972 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.pom 2019-12-09T09:43:36.972 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.pom 2019-12-09T09:43:36.972 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.pom 2019-12-09T09:43:36.972 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.pom 2019-12-09T09:43:36.972 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.pom 2019-12-09T09:43:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.pom 2019-12-09T09:43:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.pom 2019-12-09T09:43:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.pom 2019-12-09T09:43:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.pom 2019-12-09T09:43:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.pom 2019-12-09T09:43:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.pom 2019-12-09T09:43:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.pom 2019-12-09T09:43:36.974 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.pom 2019-12-09T09:43:36.974 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.pom 2019-12-09T09:43:36.974 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.pom 2019-12-09T09:43:36.974 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.pom 2019-12-09T09:43:36.974 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.pom 2019-12-09T09:43:36.975 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.pom 2019-12-09T09:43:36.975 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.pom 2019-12-09T09:43:36.975 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.pom 2019-12-09T09:43:36.975 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.pom 2019-12-09T09:43:36.975 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.pom 2019-12-09T09:43:36.975 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.pom 2019-12-09T09:43:36.976 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.pom 2019-12-09T09:43:36.976 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.pom 2019-12-09T09:43:36.976 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.pom 2019-12-09T09:43:36.976 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.pom 2019-12-09T09:43:36.977 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.pom 2019-12-09T09:43:36.977 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.pom 2019-12-09T09:43:37.071 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.pom 2019-12-09T09:43:37.071 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.pom 2019-12-09T09:43:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.pom 2019-12-09T09:43:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.pom 2019-12-09T09:43:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.pom 2019-12-09T09:43:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.pom 2019-12-09T09:43:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.pom 2019-12-09T09:43:37.072 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.pom 2019-12-09T09:43:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.pom 2019-12-09T09:43:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.pom 2019-12-09T09:43:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.pom 2019-12-09T09:43:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.pom 2019-12-09T09:43:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.pom 2019-12-09T09:43:37.073 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.pom 2019-12-09T09:43:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.pom 2019-12-09T09:43:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.pom 2019-12-09T09:43:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.pom 2019-12-09T09:43:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.pom 2019-12-09T09:43:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.pom 2019-12-09T09:43:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.pom 2019-12-09T09:43:37.074 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.pom 2019-12-09T09:43:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.pom 2019-12-09T09:43:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.pom 2019-12-09T09:43:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.pom 2019-12-09T09:43:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.pom 2019-12-09T09:43:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.pom 2019-12-09T09:43:37.075 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.pom 2019-12-09T09:43:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.pom 2019-12-09T09:43:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.pom 2019-12-09T09:43:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.pom 2019-12-09T09:43:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.pom 2019-12-09T09:43:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.pom 2019-12-09T09:43:37.076 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.22/aws-java-sdk-pom-1.11.22.pom 2019-12-09T09:43:37.166 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.pom 2019-12-09T09:43:37.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.pom 2019-12-09T09:43:37.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.pom 2019-12-09T09:43:37.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.pom 2019-12-09T09:43:37.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.pom 2019-12-09T09:43:37.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.pom 2019-12-09T09:43:37.169 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.pom 2019-12-09T09:43:37.170 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.pom 2019-12-09T09:43:37.170 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.pom 2019-12-09T09:43:37.170 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom 2019-12-09T09:43:37.170 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.pom 2019-12-09T09:43:37.170 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/project/6.1.9/project-6.1.9.pom 2019-12-09T09:43:37.170 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom 2019-12-09T09:43:37.171 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.pom 2019-12-09T09:43:37.171 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.pom 2019-12-09T09:43:37.171 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.pom 2019-12-09T09:43:37.171 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom 2019-12-09T09:43:37.171 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom 2019-12-09T09:43:37.266 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom 2019-12-09T09:43:37.266 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.pom 2019-12-09T09:43:37.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/35/commons-parent-35.pom 2019-12-09T09:43:37.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2019-12-09T09:43:37.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.pom 2019-12-09T09:43:37.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.9/httpcomponents-core-4.4.9.pom 2019-12-09T09:43:37.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/9/httpcomponents-parent-9.pom 2019-12-09T09:43:37.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.pom 2019-12-09T09:43:37.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.pom 2019-12-09T09:43:37.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom 2019-12-09T09:43:37.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/23/oss-parent-23.pom 2019-12-09T09:43:37.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.pom 2019-12-09T09:43:37.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.pom 2019-12-09T09:43:37.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-parent/4.1.17.Final/netty-parent-4.1.17.Final.pom 2019-12-09T09:43:37.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2019-12-09T09:43:37.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.pom 2019-12-09T09:43:37.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.pom 2019-12-09T09:43:37.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.pom 2019-12-09T09:43:37.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.pom 2019-12-09T09:43:37.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.pom 2019-12-09T09:43:37.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.pom 2019-12-09T09:43:37.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.pom 2019-12-09T09:43:37.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/6/oss-parent-6.pom 2019-12-09T09:43:37.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.pom 2019-12-09T09:43:37.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.pom 2019-12-09T09:43:37.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.pom 2019-12-09T09:43:37.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.pom 2019-12-09T09:43:37.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.pom 2019-12-09T09:43:37.468 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom 2019-12-09T09:43:37.469 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom 2019-12-09T09:43:37.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom 2019-12-09T09:43:37.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/17/commons-parent-17.pom 2019-12-09T09:43:37.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom 2019-12-09T09:43:37.572 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/7/apache-7.pom 2019-12-09T09:43:37.572 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pom 2019-12-09T09:43:37.572 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pom 2019-12-09T09:43:37.572 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.pom 2019-12-09T09:43:37.572 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2019-12-09T09:43:37.572 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.pom 2019-12-09T09:43:37.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.pom 2019-12-09T09:43:37.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.pom 2019-12-09T09:43:37.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom 2019-12-09T09:43:37.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.pom 2019-12-09T09:43:37.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.pom 2019-12-09T09:43:37.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom 2019-12-09T09:43:37.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.pom 2019-12-09T09:43:37.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.pom 2019-12-09T09:43:37.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom 2019-12-09T09:43:37.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom 2019-12-09T09:43:37.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.pom 2019-12-09T09:43:37.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.pom 2019-12-09T09:43:37.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.pom 2019-12-09T09:43:37.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.pom 2019-12-09T09:43:37.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.3.3/httpcomponents-client-4.3.3.pom 2019-12-09T09:43:37.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.pom 2019-12-09T09:43:37.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/9/commons-parent-9.pom 2019-12-09T09:43:37.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/4/apache-4.pom 2019-12-09T09:43:37.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/project/7/project-7.pom 2019-12-09T09:43:37.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/33/commons-parent-33.pom 2019-12-09T09:43:37.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/8.1.15.v20140411/jetty-project-8.1.15.v20140411.pom 2019-12-09T09:43:37.671 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/20/jetty-parent-20.pom 2019-12-09T09:43:37.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.pom 2019-12-09T09:43:37.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom 2019-12-09T09:43:37.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.pom 2019-12-09T09:43:37.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.pom 2019-12-09T09:43:37.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.pom 2019-12-09T09:43:37.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.pom 2019-12-09T09:43:38.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.jar 2019-12-09T09:43:38.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.jar 2019-12-09T09:43:38.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.jar 2019-12-09T09:43:38.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.jar 2019-12-09T09:43:38.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.jar 2019-12-09T09:43:38.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.jar 2019-12-09T09:43:38.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.jar 2019-12-09T09:43:38.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.jar 2019-12-09T09:43:38.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.jar 2019-12-09T09:43:38.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.jar 2019-12-09T09:43:38.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.jar 2019-12-09T09:43:38.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.jar 2019-12-09T09:43:38.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.jar 2019-12-09T09:43:38.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.jar 2019-12-09T09:43:38.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.jar 2019-12-09T09:43:38.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.jar 2019-12-09T09:43:38.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.jar 2019-12-09T09:43:38.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.jar 2019-12-09T09:43:38.570 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.jar 2019-12-09T09:43:38.570 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.jar 2019-12-09T09:43:38.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.jar 2019-12-09T09:43:38.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.jar 2019-12-09T09:43:38.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.jar 2019-12-09T09:43:38.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.jar 2019-12-09T09:43:38.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.jar 2019-12-09T09:43:38.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.jar 2019-12-09T09:43:38.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.jar 2019-12-09T09:43:38.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.jar 2019-12-09T09:43:38.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.jar 2019-12-09T09:43:38.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.jar 2019-12-09T09:43:38.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.jar 2019-12-09T09:43:38.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.jar 2019-12-09T09:43:38.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.jar 2019-12-09T09:43:38.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.jar 2019-12-09T09:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.jar 2019-12-09T09:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.jar 2019-12-09T09:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.jar 2019-12-09T09:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.jar 2019-12-09T09:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.jar 2019-12-09T09:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.jar 2019-12-09T09:43:38.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.jar 2019-12-09T09:43:38.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.jar 2019-12-09T09:43:38.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.jar 2019-12-09T09:43:38.670 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.jar 2019-12-09T09:43:38.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.jar 2019-12-09T09:43:38.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.jar 2019-12-09T09:43:38.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.jar 2019-12-09T09:43:38.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.jar 2019-12-09T09:43:38.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.jar 2019-12-09T09:43:38.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.jar 2019-12-09T09:43:38.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.jar 2019-12-09T09:43:38.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.jar 2019-12-09T09:43:38.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.jar 2019-12-09T09:43:38.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.jar 2019-12-09T09:43:38.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.jar 2019-12-09T09:43:38.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.jar 2019-12-09T09:43:38.770 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.jar 2019-12-09T09:43:38.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.jar 2019-12-09T09:43:38.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.jar 2019-12-09T09:43:38.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.jar 2019-12-09T09:43:38.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.jar 2019-12-09T09:43:38.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.jar 2019-12-09T09:43:38.771 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.jar 2019-12-09T09:43:38.772 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.jar 2019-12-09T09:43:38.772 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.jar 2019-12-09T09:43:38.772 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.jar 2019-12-09T09:43:38.772 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.jar 2019-12-09T09:43:38.866 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.jar 2019-12-09T09:43:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.jar 2019-12-09T09:43:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.jar 2019-12-09T09:43:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.jar 2019-12-09T09:43:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.jar 2019-12-09T09:43:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.jar 2019-12-09T09:43:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.jar 2019-12-09T09:43:38.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.jar 2019-12-09T09:43:38.868 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.jar 2019-12-09T09:43:38.868 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.jar 2019-12-09T09:43:38.868 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.jar 2019-12-09T09:43:38.868 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.jar 2019-12-09T09:43:38.868 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.jar 2019-12-09T09:43:38.869 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.jar 2019-12-09T09:43:38.869 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.jar 2019-12-09T09:43:38.869 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.jar 2019-12-09T09:43:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.jar 2019-12-09T09:43:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.jar 2019-12-09T09:43:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.jar 2019-12-09T09:43:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.jar 2019-12-09T09:43:38.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.jar 2019-12-09T09:43:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.jar 2019-12-09T09:43:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.jar 2019-12-09T09:43:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar 2019-12-09T09:43:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.jar 2019-12-09T09:43:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.jar 2019-12-09T09:43:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.jar 2019-12-09T09:43:38.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.jar 2019-12-09T09:43:38.968 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.jar 2019-12-09T09:43:38.968 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.jar 2019-12-09T09:43:39.066 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.jar 2019-12-09T09:43:39.066 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.jar 2019-12-09T09:43:39.066 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.jar 2019-12-09T09:43:39.067 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.jar 2019-12-09T09:43:39.067 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.jar 2019-12-09T09:43:39.067 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.jar 2019-12-09T09:43:39.067 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.jar 2019-12-09T09:43:39.068 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.jar 2019-12-09T09:43:39.068 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.jar 2019-12-09T09:43:39.068 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.jar 2019-12-09T09:43:39.166 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.jar 2019-12-09T09:43:39.166 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.jar 2019-12-09T09:43:39.166 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.jar 2019-12-09T09:43:39.166 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.jar 2019-12-09T09:43:39.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.jar 2019-12-09T09:43:39.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.jar 2019-12-09T09:43:39.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.jar 2019-12-09T09:43:39.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.jar 2019-12-09T09:43:39.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.jar 2019-12-09T09:43:39.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.jar 2019-12-09T09:43:39.167 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.jar 2019-12-09T09:43:39.168 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.jar 2019-12-09T09:43:39.168 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.jar 2019-12-09T09:43:39.168 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.jar 2019-12-09T09:43:39.168 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.jar 2019-12-09T09:43:39.266 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.jar 2019-12-09T09:43:39.266 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.jar 2019-12-09T09:43:39.266 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.jar 2019-12-09T09:43:39.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.jar 2019-12-09T09:43:39.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.jar 2019-12-09T09:43:39.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.jar 2019-12-09T09:43:39.267 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.jar 2019-12-09T09:43:39.268 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.jar 2019-12-09T09:43:39.268 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.jar 2019-12-09T09:43:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.jar 2019-12-09T09:43:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.jar 2019-12-09T09:43:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.jar 2019-12-09T09:43:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.jar 2019-12-09T09:43:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.jar 2019-12-09T09:43:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.jar 2019-12-09T09:43:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.jar 2019-12-09T09:43:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.jar 2019-12-09T09:43:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.jar 2019-12-09T09:43:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.jar 2019-12-09T09:43:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.jar 2019-12-09T09:43:39.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.jar 2019-12-09T09:43:39.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.jar 2019-12-09T09:43:39.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.jar 2019-12-09T09:43:39.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.jar 2019-12-09T09:43:39.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.jar 2019-12-09T09:43:39.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.jar 2019-12-09T09:43:39.368 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.jar 2019-12-09T09:43:39.369 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.jar 2019-12-09T09:43:39.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.jar 2019-12-09T09:43:39.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.jar 2019-12-09T09:43:39.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.jar 2019-12-09T09:43:39.466 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.jar 2019-12-09T09:43:39.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.jar 2019-12-09T09:43:39.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.jar 2019-12-09T09:43:39.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.jar 2019-12-09T09:43:39.467 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.jar 2019-12-09T09:43:39.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.jar 2019-12-09T09:43:39.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.jar 2019-12-09T09:43:39.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.jar 2019-12-09T09:43:39.568 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.jar 2019-12-09T09:43:39.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.jar 2019-12-09T09:43:39.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.jar 2019-12-09T09:43:39.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.jar 2019-12-09T09:43:39.569 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.jar 2019-12-09T09:43:39.570 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.jar 2019-12-09T09:43:39.570 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.jar 2019-12-09T09:43:39.570 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.jar 2019-12-09T09:43:39.570 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.jar 2019-12-09T09:43:39.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar 2019-12-09T09:43:39.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.jar 2019-12-09T09:43:39.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.jar 2019-12-09T09:43:39.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.jar 2019-12-09T09:43:39.571 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.jar 2019-12-09T09:43:39.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.jar 2019-12-09T09:43:39.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.jar 2019-12-09T09:43:39.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.jar 2019-12-09T09:43:39.666 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.jar 2019-12-09T09:43:39.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.jar 2019-12-09T09:43:39.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.jar 2019-12-09T09:43:39.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar 2019-12-09T09:43:39.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar 2019-12-09T09:43:39.667 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar 2019-12-09T09:43:39.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.jar 2019-12-09T09:43:39.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar 2019-12-09T09:43:39.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar 2019-12-09T09:43:39.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar 2019-12-09T09:43:39.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.jar 2019-12-09T09:43:39.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar 2019-12-09T09:43:39.668 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.jar 2019-12-09T09:43:39.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.jar 2019-12-09T09:43:39.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.jar 2019-12-09T09:43:39.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.jar 2019-12-09T09:43:39.669 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.jar 2019-12-09T09:43:39.766 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.jar 2019-12-09T09:43:39.766 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar 2019-12-09T09:43:39.766 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.jar 2019-12-09T09:43:39.766 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.jar 2019-12-09T09:43:39.766 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.jar 2019-12-09T09:43:39.767 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.jar 2019-12-09T09:43:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar 2019-12-09T09:43:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.jar 2019-12-09T09:43:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.jar 2019-12-09T09:43:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar 2019-12-09T09:43:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar 2019-12-09T09:43:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jar 2019-12-09T09:43:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.jar 2019-12-09T09:43:39.769 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.jar 2019-12-09T09:43:39.866 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.jar 2019-12-09T09:43:39.866 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar 2019-12-09T09:43:39.866 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.jar 2019-12-09T09:43:39.866 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar 2019-12-09T09:43:39.866 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.jar 2019-12-09T09:43:39.867 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.jar 2019-12-09T09:43:39.868 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.jar 2019-12-09T09:43:39.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.jar 2019-12-09T09:43:39.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar 2019-12-09T09:43:39.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.jar 2019-12-09T09:43:39.966 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.jar 2019-12-09T09:43:39.967 INFO:teuthology.orchestra.run.smithi033.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.jar 2019-12-09T09:43:48.866 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :compileJava 2019-12-09T09:43:48.866 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :processResources 2019-12-09T09:43:48.866 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :classes 2019-12-09T09:43:49.467 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:49.467 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :compileTestJava 2019-12-09T09:43:49.467 INFO:teuthology.orchestra.run.smithi033.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T09:43:49.467 INFO:teuthology.orchestra.run.smithi033.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T09:43:49.467 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:49.468 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T09:43:49.468 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :testClasses 2019-12-09T09:43:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:57.066 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :test 2019-12-09T09:43:57.066 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:43:57.066 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2019-12-09T09:44:41.464 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:44:41.465 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2019-12-09T09:44:47.564 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:44:47.565 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2019-12-09T09:44:50.464 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:44:50.464 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2019-12-09T09:44:56.664 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:44:56.664 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2019-12-09T09:45:16.664 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:45:16.664 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2019-12-09T09:45:23.764 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:45:23.764 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T09:45:24.264 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:45:24.264 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T09:45:24.864 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:45:24.864 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2019-12-09T09:45:25.639 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:16:47.116 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.657 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:16:47.134 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.662 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:16:47.139 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.679 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:16:47.156 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.685 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:16:47.185 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.706 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:16:47.205 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.853 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:16:47.330 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.878 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:16:47.355 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:25.933 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:16:47.432 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:45:28.364 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:45:28.364 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2019-12-09T09:45:35.564 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:45:35.564 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2019-12-09T09:46:07.313 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:17:28.791 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.341 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:17:28.819 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.345 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:17:28.823 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.346 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:17:28.824 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.352 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:17:28.853 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.356 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:17:28.857 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.381 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:17:28.859 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.384 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:17:28.862 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:07.390 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:17:28.891 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:46:09.563 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:46:09.563 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2019-12-09T09:46:24.763 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:46:24.763 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2019-12-09T09:50:02.760 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:50:02.761 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2019-12-09T09:50:17.160 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:50:17.160 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2019-12-09T09:50:19.260 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:50:19.260 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2019-12-09T09:50:25.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:50:25.060 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2019-12-09T09:50:30.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:50:30.060 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2019-12-09T09:50:37.348 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:21:58.829 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.387 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:21:58.868 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.388 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:21:58.869 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.394 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:21:58.875 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.438 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:21:58.942 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.444 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:21:58.949 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.553 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:21:59.034 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.573 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:21:59.054 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:37.600 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:21:59.105 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:50:39.759 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:50:39.760 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2019-12-09T09:50:55.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:50:55.060 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2019-12-09T09:51:01.659 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:51:01.660 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T09:51:02.060 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:51:02.060 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T09:51:03.259 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:51:03.259 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2019-12-09T09:51:23.159 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:51:23.160 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2019-12-09T09:51:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:51:23.660 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests STANDARD_OUT 2019-12-09T09:51:23.660 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09 09:51:23,580 [s3-transfer-manager-worker-2] WARN com.amazonaws.services.s3.internal.S3AbortableInputStream - Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use. 2019-12-09T09:51:29.459 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:51:29.460 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T09:51:50.759 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:51:50.759 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T09:52:03.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:52:03.060 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2019-12-09T09:52:33.659 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:52:33.659 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2019-12-09T09:52:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T09:52:41.559 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2019-12-09T09:54:28.841 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:25:50.320 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:28.861 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:25:50.333 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:28.889 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:25:50.368 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:28.899 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:25:50.378 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:28.921 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:25:50.413 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:28.930 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:25:50.423 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:28.984 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:25:50.463 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:29.032 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:25:50.510 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T09:54:29.131 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:25:50.624 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:00:43.076 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:00:43.076 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2019-12-09T10:00:47.076 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:00:47.077 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2019-12-09T10:01:00.048 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:32:21.512 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.057 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:32:21.521 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.068 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:32:21.533 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.076 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:32:21.540 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.076 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:32:21.562 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.084 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:32:21.570 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.316 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:32:21.781 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.343 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:32:21.807 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:00.402 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:32:21.888 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:01:29.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:01:29.678 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2019-12-09T10:01:35.677 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:01:35.677 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2019-12-09T10:02:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:02:12.278 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2019-12-09T10:02:19.778 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:02:19.778 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2019-12-09T10:02:27.478 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:02:27.478 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2019-12-09T10:02:35.178 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:02:35.178 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2019-12-09T10:02:37.778 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:02:37.778 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2019-12-09T10:03:44.779 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:03:44.779 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2019-12-09T10:03:49.079 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:03:49.079 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2019-12-09T10:03:56.479 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:03:56.479 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2019-12-09T10:03:59.679 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:03:59.679 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2019-12-09T10:04:07.679 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:07.679 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2019-12-09T10:04:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:11.479 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2019-12-09T10:04:17.079 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:17.079 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2019-12-09T10:04:20.779 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:20.779 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2019-12-09T10:04:29.579 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:29.579 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2019-12-09T10:04:32.379 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:32.379 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2019-12-09T10:04:39.479 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:39.479 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2019-12-09T10:04:42.379 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:42.379 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2019-12-09T10:04:47.279 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:04:47.279 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2019-12-09T10:05:17.479 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:05:17.479 INFO:teuthology.orchestra.run.smithi033.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2019-12-09T10:05:25.279 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:05:25.279 INFO:teuthology.orchestra.run.smithi033.stdout:---------------------------------------------------------------------- 2019-12-09T10:05:25.280 INFO:teuthology.orchestra.run.smithi033.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2019-12-09T10:05:25.280 INFO:teuthology.orchestra.run.smithi033.stdout:---------------------------------------------------------------------- 2019-12-09T10:05:25.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:05:25.328 INFO:teuthology.orchestra.run.smithi033.stdout:BUILD SUCCESSFUL in 21m 55s 2019-12-09T10:05:25.328 INFO:teuthology.orchestra.run.smithi033.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2019-12-09T10:05:25.645 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:05:25.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.844+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:25.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.875+0000 7fc778b18ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.875+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.884+0000 7fc778b18ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.884+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.892+0000 7fc778b18ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.892+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:25.912+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:26.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.126+0000 7fc778b18ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:05:26.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.126+0000 7fc778b18ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:05:26.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.126+0000 7fc778b18ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:05:26.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.126+0000 7fc778b18ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:05:26.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.126+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.149 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.147+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:26.149 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.147+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.162 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.161+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:05:26.163 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.161+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.179+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.179+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.198+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:05:26.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.198+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.209+0000 7fc778b18ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:26.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.209+0000 7fc778b18ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:05:26.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.209+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.219+0000 7fc778b18ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:26.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.219+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.237 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.236+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:26.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.236+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.255+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:05:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.255+0000 7fc778b18ec0 20 zonegroup default 2019-12-09T10:05:26.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.255+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.268+0000 7fc778b18ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.268+0000 7fc778b18ec0 10 Cannot find current period zone using local zone 2019-12-09T10:05:26.270 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.268+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.280+0000 7fc778b18ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.280+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.295+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.295+0000 7fc778b18ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.308+0000 7fc778b18ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:05:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.308+0000 7fc778b18ec0 20 zone default found 2019-12-09T10:05:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.308+0000 7fc778b18ec0 20 started sync module instance, tier type = 2019-12-09T10:05:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.308+0000 7fc778b18ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:05:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.486+0000 7fc765d42700 0 -- 172.21.15.33:0/3585673548 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x565333a29340 msgr2=0x565333a2b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:05:26.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.713+0000 7fc778b18ec0 20 add_watcher() i=0 2019-12-09T10:05:26.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.713+0000 7fc778b18ec0 20 add_watcher() i=1 2019-12-09T10:05:26.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.713+0000 7fc778b18ec0 20 add_watcher() i=2 2019-12-09T10:05:26.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.713+0000 7fc778b18ec0 20 add_watcher() i=3 2019-12-09T10:05:26.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.713+0000 7fc778b18ec0 20 add_watcher() i=4 2019-12-09T10:05:26.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.713+0000 7fc778b18ec0 20 add_watcher() i=5 2019-12-09T10:05:26.716 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.713+0000 7fc778b18ec0 20 add_watcher() i=6 2019-12-09T10:05:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.744+0000 7fc778b18ec0 20 add_watcher() i=7 2019-12-09T10:05:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.744+0000 7fc778b18ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:05:26.746 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.744+0000 7fc7367dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:05:26.757 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:26.755+0000 7fc766543700 0 -- 172.21.15.33:0/393807564 >> v2:172.21.15.33:3300/0 conn(0x565333a550e0 msgr2=0x565333a54470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:05:28.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.394+0000 7fc6acff9700 20 reqs_thread_entry: start 2019-12-09T10:05:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.394+0000 7fc6a67fc700 20 reqs_thread_entry: start 2019-12-09T10:05:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.394+0000 7fc778b18ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:05:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.394+0000 7fc6a4ff9700 20 reqs_thread_entry: start 2019-12-09T10:05:28.397 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.395+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:05:28.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.453+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:28.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.485+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.510+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.510+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.601', time=2019-12-09T11:56:33.934903+0000, chain.objs.size()=21 2019-12-09T10:05:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.510+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.1 2019-12-09T10:05:28.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.510+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.1_1 2019-12-09T10:05:28.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.510+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.2 2019-12-09T10:05:28.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.510+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.2_1 2019-12-09T10:05:28.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.3 2019-12-09T10:05:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.3_1 2019-12-09T10:05:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.4 2019-12-09T10:05:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.4_1 2019-12-09T10:05:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.5 2019-12-09T10:05:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.5_1 2019-12-09T10:05:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.6 2019-12-09T10:05:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.511+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.6_1 2019-12-09T10:05:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.732+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.7 2019-12-09T10:05:28.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.732+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.7_1 2019-12-09T10:05:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.923+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.8 2019-12-09T10:05:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.923+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.8_1 2019-12-09T10:05:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.923+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.9 2019-12-09T10:05:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.923+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.9_1 2019-12-09T10:05:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.923+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.10 2019-12-09T10:05:28.926 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.923+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.10_1 2019-12-09T10:05:28.926 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:28.923+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~KvKJZ3XN9ycWGLVW4uWfPjfrdSHI-Yl.11 2019-12-09T10:05:29.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.178+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:29.302 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.299+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:05:29.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.342+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:29.359 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.357+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:29.382 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:05:29.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.454', time=2019-12-09T11:49:15.514212+0000, chain.objs.size()=9 2019-12-09T10:05:29.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.1 2019-12-09T10:05:29.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.1_1 2019-12-09T10:05:29.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.2 2019-12-09T10:05:29.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.2_1 2019-12-09T10:05:29.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.3 2019-12-09T10:05:29.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.3_1 2019-12-09T10:05:29.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.4 2019-12-09T10:05:29.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.4_1 2019-12-09T10:05:29.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~PNjobbqTNHfvXsi6OfHTuNeQ42CDFIh.5 2019-12-09T10:05:29.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.613', time=2019-12-09T11:57:42.364543+0000, chain.objs.size()=19 2019-12-09T10:05:29.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.1 2019-12-09T10:05:29.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.1_1 2019-12-09T10:05:29.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.380+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.2 2019-12-09T10:05:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.681+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.2_1 2019-12-09T10:05:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.682+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.3 2019-12-09T10:05:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.682+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.3_1 2019-12-09T10:05:29.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.682+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.4 2019-12-09T10:05:29.685 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.682+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.4_1 2019-12-09T10:05:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.5 2019-12-09T10:05:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.5_1 2019-12-09T10:05:29.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.6 2019-12-09T10:05:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.6_1 2019-12-09T10:05:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.7 2019-12-09T10:05:29.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.7_1 2019-12-09T10:05:29.868 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:29.866+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.8 2019-12-09T10:05:30.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.058+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.8_1 2019-12-09T10:05:30.060 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.058+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.9 2019-12-09T10:05:30.060 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.058+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.9_1 2019-12-09T10:05:30.060 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.058+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~Zc-kampw4JZ_PaAUbQGRVAucFfKhHC5.10 2019-12-09T10:05:30.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.286+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:30.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.410+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:05:30.470 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.468+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:30.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.494+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.466', time=2019-12-09T11:50:01.287946+0000, chain.objs.size()=7 2019-12-09T10:05:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~Iy1J7WOMZ0jhz40qaXdZ4bSey1L0SH3.1 2019-12-09T10:05:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~Iy1J7WOMZ0jhz40qaXdZ4bSey1L0SH3.1_1 2019-12-09T10:05:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~Iy1J7WOMZ0jhz40qaXdZ4bSey1L0SH3.1_2 2019-12-09T10:05:30.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~Iy1J7WOMZ0jhz40qaXdZ4bSey1L0SH3.2 2019-12-09T10:05:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~Iy1J7WOMZ0jhz40qaXdZ4bSey1L0SH3.2_1 2019-12-09T10:05:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~Iy1J7WOMZ0jhz40qaXdZ4bSey1L0SH3.2_2 2019-12-09T10:05:30.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.514+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~Iy1J7WOMZ0jhz40qaXdZ4bSey1L0SH3.3 2019-12-09T10:05:30.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.841+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:30.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:30.964+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:05:31.025 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.023+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:31.045 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.044+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:31.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.417', time=2019-12-09T11:46:11.667146+0000, chain.objs.size()=9 2019-12-09T10:05:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.1 2019-12-09T10:05:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.1_1 2019-12-09T10:05:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.2 2019-12-09T10:05:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.2_1 2019-12-09T10:05:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.3 2019-12-09T10:05:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.3_1 2019-12-09T10:05:31.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.4 2019-12-09T10:05:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.4_1 2019-12-09T10:05:31.063 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.059+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.5 2019-12-09T10:05:31.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.273+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:31.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.368+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:05:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.412+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:31.430 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.429+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.440+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.440+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.430', time=2019-12-09T11:50:05.615197+0000, chain.objs.size()=5 2019-12-09T10:05:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.440+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.19__shadow_.ukYgKSiOLNjX1-YI7T3Trc7pk3Wkzc6_1 2019-12-09T10:05:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.440+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.19__shadow_.ukYgKSiOLNjX1-YI7T3Trc7pk3Wkzc6_2 2019-12-09T10:05:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.440+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.19__shadow_.ukYgKSiOLNjX1-YI7T3Trc7pk3Wkzc6_3 2019-12-09T10:05:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.440+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.19__shadow_.ukYgKSiOLNjX1-YI7T3Trc7pk3Wkzc6_4 2019-12-09T10:05:31.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.440+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.19__shadow_.ukYgKSiOLNjX1-YI7T3Trc7pk3Wkzc6_5 2019-12-09T10:05:31.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.606+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:31.741 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.739+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:05:31.786 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.785+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:31.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.801+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:31.811 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.809+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.858+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:05:31.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.920+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:31.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.957+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:31.981 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.667', time=2019-12-09T12:02:14.855494+0000, chain.objs.size()=9 2019-12-09T10:05:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__multipart_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.1 2019-12-09T10:05:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__shadow_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.1_1 2019-12-09T10:05:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__multipart_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.2 2019-12-09T10:05:31.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__shadow_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.2_1 2019-12-09T10:05:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__multipart_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.3 2019-12-09T10:05:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.979+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__shadow_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.3_1 2019-12-09T10:05:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.980+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__multipart_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.4 2019-12-09T10:05:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.980+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__shadow_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.4_1 2019-12-09T10:05:31.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:31.980+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.30__multipart_key1.2~ICfWwi3epbZ9fomhFW5MONBY8Vj8v7Z.5 2019-12-09T10:05:32.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.279+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:32.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.366+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:05:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.414+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:32.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.429+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.445+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.621', time=2019-12-09T11:59:35.777209+0000, chain.objs.size()=16 2019-12-09T10:05:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.1 2019-12-09T10:05:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.1_1 2019-12-09T10:05:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.1_2 2019-12-09T10:05:32.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.2 2019-12-09T10:05:32.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.2_1 2019-12-09T10:05:32.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.2_2 2019-12-09T10:05:32.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.3 2019-12-09T10:05:32.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.3_1 2019-12-09T10:05:32.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.3_2 2019-12-09T10:05:32.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.4 2019-12-09T10:05:32.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.4_1 2019-12-09T10:05:32.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.446+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.4_2 2019-12-09T10:05:32.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.609+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.5 2019-12-09T10:05:32.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.609+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.5_1 2019-12-09T10:05:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.701+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.5_2 2019-12-09T10:05:32.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:32.701+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~SwyZ2-jHNMfCKG86GdH5nOtHR0HGh4T.6 2019-12-09T10:05:33.042 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.040+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.161+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:05:33.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.213+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:33.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.234+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:05:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.415', time=2019-12-09T11:46:18.682851+0000, chain.objs.size()=9 2019-12-09T10:05:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.1 2019-12-09T10:05:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.1_1 2019-12-09T10:05:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.2 2019-12-09T10:05:33.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.2_1 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.3 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.3_1 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.4 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__shadow_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.4_1 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.17__multipart_key1.2~fln3OyMUSiKh05ysCC6Ns06M4V8LCtd.5 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.472', time=2019-12-09T11:50:11.534135+0000, chain.objs.size()=7 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~bU8vA1ykkq2V-NydP2L5sfocRTaVnqB.1 2019-12-09T10:05:33.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~bU8vA1ykkq2V-NydP2L5sfocRTaVnqB.1_1 2019-12-09T10:05:33.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.253+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~bU8vA1ykkq2V-NydP2L5sfocRTaVnqB.1_2 2019-12-09T10:05:33.379 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.377+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~bU8vA1ykkq2V-NydP2L5sfocRTaVnqB.2 2019-12-09T10:05:33.379 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.377+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~bU8vA1ykkq2V-NydP2L5sfocRTaVnqB.2_1 2019-12-09T10:05:33.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.448+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~bU8vA1ykkq2V-NydP2L5sfocRTaVnqB.2_2 2019-12-09T10:05:33.484 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.482+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~bU8vA1ykkq2V-NydP2L5sfocRTaVnqB.3 2019-12-09T10:05:33.679 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.677+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:33.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.808+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:05:33.857 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.855+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:33.880 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.878+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:33.894 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.892+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.932+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:05:33.992 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:33.991+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:34.011 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.009+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:34.030 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.386', time=2019-12-09T11:45:18.594943+0000, chain.objs.size()=9 2019-12-09T10:05:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__multipart_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.1 2019-12-09T10:05:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__shadow_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.1_1 2019-12-09T10:05:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__multipart_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.2 2019-12-09T10:05:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__shadow_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.2_1 2019-12-09T10:05:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__multipart_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.3 2019-12-09T10:05:34.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__shadow_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.3_1 2019-12-09T10:05:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__multipart_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.4 2019-12-09T10:05:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__shadow_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.4_1 2019-12-09T10:05:34.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.028+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.15__multipart_key1.2~TYHieEQYBf2HVF03o1SM5jg36p8n8fA.5 2019-12-09T10:05:34.335 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.334+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:34.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.462+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:05:34.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.510+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:34.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.528+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:34.540 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.539+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.584+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:05:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.662+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:34.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.680+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:34.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.460', time=2019-12-09T11:49:38.379068+0000, chain.objs.size()=7 2019-12-09T10:05:34.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~0qv5unqFyfEuP-SUGN7ZU7xW0a-Xmzw.1 2019-12-09T10:05:34.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~0qv5unqFyfEuP-SUGN7ZU7xW0a-Xmzw.1_1 2019-12-09T10:05:34.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~0qv5unqFyfEuP-SUGN7ZU7xW0a-Xmzw.1_2 2019-12-09T10:05:34.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~0qv5unqFyfEuP-SUGN7ZU7xW0a-Xmzw.2 2019-12-09T10:05:34.699 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~0qv5unqFyfEuP-SUGN7ZU7xW0a-Xmzw.2_1 2019-12-09T10:05:34.699 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~0qv5unqFyfEuP-SUGN7ZU7xW0a-Xmzw.2_2 2019-12-09T10:05:34.699 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.695+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~0qv5unqFyfEuP-SUGN7ZU7xW0a-Xmzw.3 2019-12-09T10:05:34.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:34.922+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:35.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.026+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:05:35.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.069+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:35.086 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.085+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:35.099 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.098+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:35.143 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.142+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:05:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.201+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:35.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.220+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:35.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.232+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.281+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:05:35.364 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.362+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.390+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:35.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.398+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:35.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.447+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:05:35.521 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.519+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.541+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.553', time=2019-12-09T11:51:53.079246+0000, chain.objs.size()=9 2019-12-09T10:05:35.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__multipart_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.1 2019-12-09T10:05:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__shadow_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.1_1 2019-12-09T10:05:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__multipart_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.2 2019-12-09T10:05:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__shadow_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.2_1 2019-12-09T10:05:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__multipart_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.3 2019-12-09T10:05:35.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__shadow_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.3_1 2019-12-09T10:05:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__multipart_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.4 2019-12-09T10:05:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__shadow_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.4_1 2019-12-09T10:05:35.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.557+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__multipart_file.mpg.2~RA_SYYBWNSVUO3vAJG23Grp1TVsTvAA.5 2019-12-09T10:05:35.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:35.943+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:36.039 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.038+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:05:36.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.110+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:36.136 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.134+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:36.149 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.147+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:36.198 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.196+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:05:36.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.281+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.311+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:36.354 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.352+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.352+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~araVYbxqCZTIwpnVjEjIgF4zUWV1kDs', time=2019-12-09T11:52:37.100422+0000, chain.objs.size()=5 2019-12-09T10:05:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.352+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.27__multipart_key1.2~araVYbxqCZTIwpnVjEjIgF4zUWV1kDs.7 2019-12-09T10:05:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.352+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.27__shadow_key1.2~araVYbxqCZTIwpnVjEjIgF4zUWV1kDs.7_1 2019-12-09T10:05:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.352+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.27__multipart_key1.2~araVYbxqCZTIwpnVjEjIgF4zUWV1kDs.10 2019-12-09T10:05:36.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.352+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.27__shadow_key1.2~araVYbxqCZTIwpnVjEjIgF4zUWV1kDs.10_1 2019-12-09T10:05:36.356 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.352+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.27__multipart_key1.2~araVYbxqCZTIwpnVjEjIgF4zUWV1kDs.13 2019-12-09T10:05:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.558+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:36.652 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.650+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:05:36.694 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.692+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.708+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:05:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m', time=2019-12-09T11:44:40.542992+0000, chain.objs.size()=9 2019-12-09T10:05:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__multipart_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.1 2019-12-09T10:05:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__shadow_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.1_1 2019-12-09T10:05:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__multipart_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.2 2019-12-09T10:05:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__shadow_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.2_1 2019-12-09T10:05:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__multipart_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.3 2019-12-09T10:05:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__shadow_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.3_1 2019-12-09T10:05:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__multipart_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.4 2019-12-09T10:05:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__shadow_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.4_1 2019-12-09T10:05:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.13__multipart_key1.2~8tRx0-eBnfKktj4NiLcfEh-PTHJBh2m.5 2019-12-09T10:05:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.528', time=2019-12-09T11:51:25.327235+0000, chain.objs.size()=9 2019-12-09T10:05:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__multipart_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.1 2019-12-09T10:05:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__shadow_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.1_1 2019-12-09T10:05:36.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.724+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__multipart_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.2 2019-12-09T10:05:36.818 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.816+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__shadow_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.2_1 2019-12-09T10:05:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.966+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__multipart_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.3 2019-12-09T10:05:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.966+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__shadow_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.3_1 2019-12-09T10:05:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.966+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__multipart_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.4 2019-12-09T10:05:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.966+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__shadow_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.4_1 2019-12-09T10:05:36.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:36.966+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.25__multipart_key1.2~GbnyqtTmIKMcPGCkpG2LhH7DWuolPJJ.5 2019-12-09T10:05:37.303 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.301+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:37.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.385+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:05:37.430 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.428+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.454+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:37.467 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.466+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.506+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:05:37.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.555+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.571+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:37.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.579+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.609+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:05:37.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.661+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.678 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.677+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:37.687 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.686+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.729+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:05:37.789 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.788+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.811 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.809+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.818+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.852+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:05:37.903 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.901+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.918+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:37.943 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:37.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6', time=2019-12-09T12:02:32.229551+0000, chain.objs.size()=23 2019-12-09T10:05:37.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.1 2019-12-09T10:05:37.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.2 2019-12-09T10:05:37.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.3 2019-12-09T10:05:37.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.4 2019-12-09T10:05:37.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.5 2019-12-09T10:05:37.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.6 2019-12-09T10:05:37.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.7 2019-12-09T10:05:37.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.8 2019-12-09T10:05:37.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.9 2019-12-09T10:05:37.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.10 2019-12-09T10:05:37.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.11 2019-12-09T10:05:37.946 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:37.941+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.gXDhfrE6gEQVShPlr5FgZfC_Fbjze9c.12 2019-12-09T10:05:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.129+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.13 2019-12-09T10:05:38.132 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.129+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.QJ0bHNZfJABUdMBH9vfvli5O3kqNA1s.14 2019-12-09T10:05:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.15 2019-12-09T10:05:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.7d7JDFd5jQ1NSs9v49BhhcN-rBMDolZ.16 2019-12-09T10:05:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.IXIXPT-B87F27NzaXKE69KsMxoyhKfb.17 2019-12-09T10:05:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.w_I02SRi8AXFDBOopuHNctD5w7SfdlI.18 2019-12-09T10:05:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.19 2019-12-09T10:05:38.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.20 2019-12-09T10:05:38.175 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.21 2019-12-09T10:05:38.175 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.22 2019-12-09T10:05:38.175 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.172+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.31__multipart_key1.2~lvFNJLHF3psrqXP0VpCkuz3KCCifZJ6.23 2019-12-09T10:05:38.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.536+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:38.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.664+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:05:38.738 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.735+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.763+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.775+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.830+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:05:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.884+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:38.912 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.910+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T10:05:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.438', time=2019-12-09T11:48:18.411616+0000, chain.objs.size()=9 2019-12-09T10:05:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.1 2019-12-09T10:05:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.1_1 2019-12-09T10:05:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.2 2019-12-09T10:05:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.2_1 2019-12-09T10:05:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.3 2019-12-09T10:05:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.3_1 2019-12-09T10:05:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.4 2019-12-09T10:05:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.4_1 2019-12-09T10:05:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~tkH1fle68qyXWt3e02ftLFVEsCbQ8F7.5 2019-12-09T10:05:38.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.546', time=2019-12-09T11:51:55.771155+0000, chain.objs.size()=3 2019-12-09T10:05:38.937 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__shadow_.NKKmQq-RwwcAVtZo-1b-sOHkqc8iv7U_1 2019-12-09T10:05:38.937 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__shadow_.NKKmQq-RwwcAVtZo-1b-sOHkqc8iv7U_2 2019-12-09T10:05:38.937 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:38.931+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.26__shadow_.NKKmQq-RwwcAVtZo-1b-sOHkqc8iv7U_3 2019-12-09T10:05:39.171 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.168+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.652', time=2019-12-09T12:01:31.543797+0000, chain.objs.size()=9 2019-12-09T10:05:39.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.168+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__multipart_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.1 2019-12-09T10:05:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.198+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__shadow_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.1_1 2019-12-09T10:05:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.198+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__multipart_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.2 2019-12-09T10:05:39.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.198+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__shadow_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.2_1 2019-12-09T10:05:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.320+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__multipart_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.3 2019-12-09T10:05:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.320+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__shadow_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.3_1 2019-12-09T10:05:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.320+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__multipart_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.4 2019-12-09T10:05:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.320+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__shadow_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.4_1 2019-12-09T10:05:39.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.320+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.29__multipart_key1.2~lGaRoqOCodon9_fJRxW6a0rgRae8jGg.5 2019-12-09T10:05:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.525+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:39.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.659+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:05:39.706 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.703+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:39.723 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.719+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:05:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.446', time=2019-12-09T11:48:50.373139+0000, chain.objs.size()=9 2019-12-09T10:05:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.1 2019-12-09T10:05:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.1_1 2019-12-09T10:05:39.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.2 2019-12-09T10:05:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.2_1 2019-12-09T10:05:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.3 2019-12-09T10:05:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.3_1 2019-12-09T10:05:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.4 2019-12-09T10:05:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__shadow_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.4_1 2019-12-09T10:05:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.20__multipart_key1.2~n0uPrD_6btvzEAz2xUrrBmpDgxP8Woh.5 2019-12-09T10:05:39.770 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.823', time=2019-12-09T12:05:19.096729+0000, chain.objs.size()=21 2019-12-09T10:05:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.1 2019-12-09T10:05:39.771 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.1_1 2019-12-09T10:05:39.772 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.765+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.2 2019-12-09T10:05:39.995 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.992+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.2_1 2019-12-09T10:05:39.995 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:39.992+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.3 2019-12-09T10:05:40.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.073+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.3_1 2019-12-09T10:05:40.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.073+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.4 2019-12-09T10:05:40.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.073+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.4_1 2019-12-09T10:05:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.5 2019-12-09T10:05:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.5_1 2019-12-09T10:05:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.6 2019-12-09T10:05:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.6_1 2019-12-09T10:05:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.7 2019-12-09T10:05:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.7_1 2019-12-09T10:05:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.8 2019-12-09T10:05:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.220+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.8_1 2019-12-09T10:05:40.330 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.327+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.9 2019-12-09T10:05:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.429+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.9_1 2019-12-09T10:05:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.429+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.10 2019-12-09T10:05:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.429+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__shadow_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.10_1 2019-12-09T10:05:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.429+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.39__multipart_key1.2~TRbduj3yT9ruswgFyb6KVFepoQJqZDX.11 2019-12-09T10:05:40.720 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.716+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.876+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:05:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.920+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:40.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.943+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.509', time=2019-12-09T11:50:56.541733+0000, chain.objs.size()=9 2019-12-09T10:05:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__multipart_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.1 2019-12-09T10:05:40.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__shadow_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.1_1 2019-12-09T10:05:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__multipart_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.2 2019-12-09T10:05:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__shadow_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.2_1 2019-12-09T10:05:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__multipart_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.3 2019-12-09T10:05:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__shadow_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.3_1 2019-12-09T10:05:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__multipart_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.4 2019-12-09T10:05:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__shadow_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.4_1 2019-12-09T10:05:40.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:40.965+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.24__multipart_key1.2~mpv0yAsTJvblcji6wLzw7MuDdru6wqq.5 2019-12-09T10:05:41.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.190+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:41.319 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.313+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:05:41.366 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.363+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.378+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:41.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:05:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.629', time=2019-12-09T12:00:41.958046+0000, chain.objs.size()=16 2019-12-09T10:05:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.1 2019-12-09T10:05:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.1_1 2019-12-09T10:05:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.1_2 2019-12-09T10:05:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.2 2019-12-09T10:05:41.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.2_1 2019-12-09T10:05:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.2_2 2019-12-09T10:05:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.3 2019-12-09T10:05:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.3_1 2019-12-09T10:05:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.3_2 2019-12-09T10:05:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.4 2019-12-09T10:05:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.4_1 2019-12-09T10:05:41.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.396+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.4_2 2019-12-09T10:05:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.641+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.5 2019-12-09T10:05:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.641+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.5_1 2019-12-09T10:05:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.641+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.5_2 2019-12-09T10:05:41.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.641+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~oblsZ-YsGu4fuZNfmUfrZtgfnOhCscE.6 2019-12-09T10:05:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:41.968+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.055+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:05:42.097 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.094+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:42.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.109+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.133+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:05:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.133+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.588', time=2019-12-09T11:55:15.145573+0000, chain.objs.size()=21 2019-12-09T10:05:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.133+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.1 2019-12-09T10:05:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.1_1 2019-12-09T10:05:42.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.2 2019-12-09T10:05:42.138 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.2_1 2019-12-09T10:05:42.138 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.3 2019-12-09T10:05:42.138 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.3_1 2019-12-09T10:05:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.4 2019-12-09T10:05:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.4_1 2019-12-09T10:05:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.5 2019-12-09T10:05:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.5_1 2019-12-09T10:05:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.6 2019-12-09T10:05:42.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.134+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.6_1 2019-12-09T10:05:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.345+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.7 2019-12-09T10:05:42.348 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.345+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.7_1 2019-12-09T10:05:42.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.397+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.8 2019-12-09T10:05:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.397+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.8_1 2019-12-09T10:05:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.397+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.9 2019-12-09T10:05:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.397+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.9_1 2019-12-09T10:05:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.397+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.10 2019-12-09T10:05:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.397+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.10_1 2019-12-09T10:05:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.470+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~r2RUh5VPPVlNRgncgi1ubLNhEO_4Hnt.11 2019-12-09T10:05:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.470+0000 7fc778b18ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.637', time=2019-12-09T12:00:44.357880+0000, chain.objs.size()=16 2019-12-09T10:05:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.470+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.1 2019-12-09T10:05:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.470+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.1_1 2019-12-09T10:05:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.508+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.1_2 2019-12-09T10:05:42.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.613+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.2 2019-12-09T10:05:42.734 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.731+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.2_1 2019-12-09T10:05:42.738 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.735+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.2_2 2019-12-09T10:05:42.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.735+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.3 2019-12-09T10:05:42.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.735+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.3_1 2019-12-09T10:05:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.735+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.3_2 2019-12-09T10:05:42.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.735+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.4 2019-12-09T10:05:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.952+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.4_1 2019-12-09T10:05:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.952+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.4_2 2019-12-09T10:05:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.952+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.5 2019-12-09T10:05:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.952+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.5_1 2019-12-09T10:05:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.952+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__shadow_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.5_2 2019-12-09T10:05:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:42.952+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.28__multipart_key1.2~1kfnaOZGtsXq2cWURhATCAQLd2EIYxQ.6 2019-12-09T10:05:43.283 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.280+0000 7fc778b18ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:05:43.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.414+0000 7fc778b18ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:05:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.462+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:43.486 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.484+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:43.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.493+0000 7fc778b18ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.569+0000 7fc778b18ec0 20 remove_watcher() i=0 2019-12-09T10:05:43.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.569+0000 7fc778b18ec0 2 removed watcher, disabling cache 2019-12-09T10:05:43.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.612+0000 7fc778b18ec0 20 remove_watcher() i=1 2019-12-09T10:05:43.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.656+0000 7fc778b18ec0 20 remove_watcher() i=2 2019-12-09T10:05:43.707 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.704+0000 7fc778b18ec0 20 remove_watcher() i=3 2019-12-09T10:05:43.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.757+0000 7fc778b18ec0 20 remove_watcher() i=4 2019-12-09T10:05:43.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.799+0000 7fc778b18ec0 20 remove_watcher() i=5 2019-12-09T10:05:43.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.852+0000 7fc778b18ec0 20 remove_watcher() i=6 2019-12-09T10:05:43.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:43.892+0000 7fc778b18ec0 20 remove_watcher() i=7 2019-12-09T10:05:43.903 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:05:44.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.012+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.047+0000 7f9e28a3fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.047+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.059+0000 7f9e28a3fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:44.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.059+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.076+0000 7f9e28a3fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:44.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.076+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.099 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.097+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:44.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.291+0000 7f9e28a3fec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:05:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.291+0000 7f9e28a3fec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:05:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.291+0000 7f9e28a3fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:05:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.291+0000 7f9e28a3fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:05:44.294 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.291+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.302+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:44.305 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.303+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.340+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:05:44.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.340+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.361+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.361+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.381+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:05:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.381+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.389+0000 7f9e28a3fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.389+0000 7f9e28a3fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:05:44.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.389+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.397+0000 7f9e28a3fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.397+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.412+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.412+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.429+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:05:44.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.429+0000 7f9e28a3fec0 20 zonegroup default 2019-12-09T10:05:44.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.429+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.438+0000 7f9e28a3fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:44.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.438+0000 7f9e28a3fec0 10 Cannot find current period zone using local zone 2019-12-09T10:05:44.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.438+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.445+0000 7f9e28a3fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:44.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.445+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.457+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:44.460 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.457+0000 7f9e28a3fec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:44.472 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.469+0000 7f9e28a3fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:05:44.472 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.469+0000 7f9e28a3fec0 20 zone default found 2019-12-09T10:05:44.472 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.469+0000 7f9e28a3fec0 20 started sync module instance, tier type = 2019-12-09T10:05:44.472 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.469+0000 7f9e28a3fec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:05:44.868 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.865+0000 7f9e28a3fec0 20 add_watcher() i=0 2019-12-09T10:05:44.868 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.865+0000 7f9e28a3fec0 20 add_watcher() i=1 2019-12-09T10:05:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.865+0000 7f9e28a3fec0 20 add_watcher() i=2 2019-12-09T10:05:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.865+0000 7f9e28a3fec0 20 add_watcher() i=3 2019-12-09T10:05:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.865+0000 7f9e28a3fec0 20 add_watcher() i=4 2019-12-09T10:05:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.865+0000 7f9e28a3fec0 20 add_watcher() i=5 2019-12-09T10:05:44.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.865+0000 7f9e28a3fec0 20 add_watcher() i=6 2019-12-09T10:05:44.903 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.900+0000 7f9e28a3fec0 20 add_watcher() i=7 2019-12-09T10:05:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.900+0000 7f9e28a3fec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:05:44.904 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.900+0000 7f9de67dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:05:44.906 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.904+0000 7f9e0ffff700 0 -- 172.21.15.33:0/1983505295 >> v1:172.21.15.186:6789/0 conn(0x55a2980fb610 legacy=0x55a2980fba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T10:05:44.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:44.949+0000 7f9e0ffff700 0 -- 172.21.15.33:0/2560596988 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x55a298134b30 msgr2=0x55a298136f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:05:46.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.597+0000 7f9d5cff9700 20 reqs_thread_entry: start 2019-12-09T10:05:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.598+0000 7f9d567fc700 20 reqs_thread_entry: start 2019-12-09T10:05:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.598+0000 7f9e28a3fec0 20 init_complete bucket index max shards: 0 2019-12-09T10:05:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.598+0000 7f9d54ff9700 20 reqs_thread_entry: start 2019-12-09T10:05:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.598+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:05:46.672 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.669+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:46.701 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.698+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.719+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:46.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.771+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:05:46.828 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.825+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:46.852 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.849+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:46.865 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.862+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.906+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:05:46.973 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.970+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:46.995 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:46.992+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:47.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.005+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.047+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:05:47.116 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.113+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.136 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.134+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:47.151 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.144+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.195 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.192+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:05:47.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.290+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.334+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:47.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.349+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.389+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:05:47.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.470+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.499 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.496+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:47.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.506+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.547+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:05:47.641 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.638+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.672 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.669+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:47.689 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.687+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.772 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.769+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:05:47.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.823+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.841+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:47.853 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.851+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.891+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:05:47.942 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.939+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:47.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.958+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:47.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:47.968+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.002+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:05:48.050 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.047+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.066 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.063+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:48.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.072+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.122 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.119+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:05:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.162+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.176+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.186+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.219+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:05:48.301 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.298+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.321 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.319+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:48.333 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.330+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.398+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:05:48.453 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.450+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.472 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.470+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:48.485 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.483+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.540 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.533+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:05:48.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.603+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.635+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:48.654 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.651+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.701 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.698+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:05:48.778 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.775+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.817 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.814+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.829+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.878+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:05:48.933 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.930+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:48.951 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.949+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:48.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.958+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.002 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:48.999+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:05:49.112 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.109+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.136+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:49.150 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.147+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.233+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:05:49.296 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.293+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.324 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.321+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:49.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.340+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.389+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:05:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.471+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.498 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.495+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:49.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.506+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.567+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:05:49.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.635+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.663 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.660+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:49.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.670+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.725+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:05:49.791 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.788+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.818 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.815+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:49.830 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.827+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.868 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.866+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:05:49.918 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.915+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.938+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.949+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:49.985 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:49.982+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:05:50.045 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.042+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.062+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:50.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.073+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.112+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:05:50.190 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.187+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.213+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:50.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.231+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.277 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.274+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:05:50.346 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.342+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.365 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.362+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:50.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.372+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.420 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.418+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:05:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.470+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.490 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.487+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:50.503 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.500+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.539 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.537+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:05:50.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.590+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.620+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:50.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.634+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.681+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:05:50.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.744+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.772 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.769+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:50.784 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.781+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.840 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.837+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:05:50.901 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.898+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.921 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.918+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:50.930 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.927+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:50.976+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:05:51.039 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.036+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:51.060 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.057+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:51.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.069+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.118+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:05:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.172+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:51.194 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.191+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:51.204 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.201+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:51.252 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.249+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:05:51.358 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.355+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.393+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:51.416 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.413+0000 7f9e28a3fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.546+0000 7f9e28a3fec0 20 remove_watcher() i=0 2019-12-09T10:05:51.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.546+0000 7f9e28a3fec0 2 removed watcher, disabling cache 2019-12-09T10:05:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.611+0000 7f9e28a3fec0 20 remove_watcher() i=1 2019-12-09T10:05:51.680 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.677+0000 7f9e28a3fec0 20 remove_watcher() i=2 2019-12-09T10:05:51.725 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.722+0000 7f9e28a3fec0 20 remove_watcher() i=3 2019-12-09T10:05:51.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.790+0000 7f9e28a3fec0 20 remove_watcher() i=4 2019-12-09T10:05:51.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.846+0000 7f9e28a3fec0 20 remove_watcher() i=5 2019-12-09T10:05:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.905+0000 7f9e28a3fec0 20 remove_watcher() i=6 2019-12-09T10:05:51.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:51.949+0000 7f9e28a3fec0 20 remove_watcher() i=7 2019-12-09T10:05:51.962 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:05:52.082 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.080+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.110 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.108+0000 7f26706a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:52.111 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.109+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.117 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.115+0000 7f26706a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:52.117 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.115+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.121+0000 7f26706a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.121+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.139+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.336+0000 7f26706a2ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:05:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.336+0000 7f26706a2ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:05:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.336+0000 7f26706a2ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:05:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.336+0000 7f26706a2ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:05:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.336+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.356 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.354+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:52.356 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.354+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.373+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:05:52.376 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.374+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.389 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.387+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:52.389 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.387+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.399+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:05:52.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.399+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.407 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.406+0000 7f26706a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.406+0000 7f26706a2ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:05:52.408 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.406+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.415 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.414+0000 7f26706a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:52.416 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.414+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.427+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.427+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.442+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:05:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.442+0000 7f26706a2ec0 20 zonegroup default 2019-12-09T10:05:52.444 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.442+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.448+0000 7f26706a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.448+0000 7f26706a2ec0 10 Cannot find current period zone using local zone 2019-12-09T10:05:52.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.448+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.457+0000 7f26706a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:05:52.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.457+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.468 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.466+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:05:52.469 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.467+0000 7f26706a2ec0 20 rados->read ofs=0 len=0 2019-12-09T10:05:52.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.485+0000 7f26706a2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:05:52.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.485+0000 7f26706a2ec0 20 zone default found 2019-12-09T10:05:52.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.485+0000 7f26706a2ec0 20 started sync module instance, tier type = 2019-12-09T10:05:52.487 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.485+0000 7f26706a2ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:05:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.820+0000 7f26706a2ec0 20 add_watcher() i=0 2019-12-09T10:05:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.820+0000 7f26706a2ec0 20 add_watcher() i=1 2019-12-09T10:05:52.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.820+0000 7f26706a2ec0 20 add_watcher() i=2 2019-12-09T10:05:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.820+0000 7f26706a2ec0 20 add_watcher() i=3 2019-12-09T10:05:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.820+0000 7f26706a2ec0 20 add_watcher() i=4 2019-12-09T10:05:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.820+0000 7f26706a2ec0 20 add_watcher() i=5 2019-12-09T10:05:52.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.820+0000 7f26706a2ec0 20 add_watcher() i=6 2019-12-09T10:05:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.861+0000 7f26706a2ec0 20 add_watcher() i=7 2019-12-09T10:05:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.861+0000 7f26706a2ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:05:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.861+0000 7f262e7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:05:52.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:52.880+0000 7f265e0cd700 0 -- >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x562ea54d3710 msgr2=0x562ea54d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:05:53.024 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:53.022+0000 7f265e0cd700 0 -- 172.21.15.33:0/2988201932 >> [v2:172.21.15.186:6810/10132,v1:172.21.15.186:6811/10132] conn(0x562ea5501a00 msgr2=0x562ea5503e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:05:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.598+0000 7f25a4ff9700 20 reqs_thread_entry: start 2019-12-09T10:05:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.598+0000 7f259e7fc700 20 reqs_thread_entry: start 2019-12-09T10:05:54.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.598+0000 7f26706a2ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:05:54.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.599+0000 7f259cff9700 20 reqs_thread_entry: start 2019-12-09T10:05:54.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.599+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:05:54.656 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.654+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:54.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.680+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:54.691 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.690+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.745+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:05:54.805 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.803+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:54.831 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.830+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:54.840 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.838+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:54.876 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.874+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:05:54.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.936+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:54.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.957+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:54.972 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:54.970+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.006 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.005+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:05:55.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.047+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.067 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.066+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:55.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.076+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.231+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:05:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.315+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.342+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:55.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.358+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.411+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:05:55.488 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.486+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.516+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:55.541 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.539+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.575+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:05:55.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.648+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.673 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.671+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:55.684 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.682+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.729+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:05:55.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.850+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.910 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.909+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.921+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:55.975 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:55.973+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:05:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.026+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.044 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.042+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:56.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.051+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.082 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.080+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:05:56.130 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.128+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.150 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.148+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:56.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.158+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.208+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:05:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.262+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.279+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.289+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.340+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:05:56.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.617+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.657 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.655+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:56.670 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.669+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.719 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.717+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:05:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.779+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.794+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:56.805 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.803+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.849+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:05:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.927+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:56.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.958+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:56.978 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:56.977+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.054+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:05:57.138 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.136+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.161+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:57.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.172+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.215+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:05:57.280 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.278+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.297+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:57.309 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.307+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.380+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:05:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.441+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.462 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.460+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:57.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.472+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.519 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.518+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:05:57.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.590+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.614+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.629+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.687 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.685+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:05:57.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.757+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.783 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.781+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:57.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.796+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.844+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:05:57.923 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.921+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.972+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:57.992 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:57.990+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.051 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.050+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:05:58.122 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.120+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.153 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.151+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:58.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.170+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.226 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.224+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:05:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.312+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.343+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:58.359 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.357+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.418 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.416+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:05:58.502 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.500+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.524 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.523+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:58.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.535+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.587+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:05:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.656+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.685 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.683+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:58.700 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.698+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.761 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.759+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:05:58.842 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.840+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.866 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.864+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:58.880 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.878+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.923+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:05:58.977 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:58.975+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.006 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.004+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:59.018 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.016+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.059+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:05:59.140 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.138+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.176+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:59.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.198+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.269+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:05:59.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.347+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.380 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.379+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:59.406 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.404+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.453 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.451+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:05:59.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.509+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.541+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:59.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.556+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.619+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:05:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.673+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.694+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:59.707 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.705+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.748+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:05:59.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.792+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.812 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.810+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:59.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.820+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.852+0000 7f26706a2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:05:59.911 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.909+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:05:59.929 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.927+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:05:59.937 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:05:59.935+0000 7f26706a2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.044+0000 7f26706a2ec0 20 remove_watcher() i=0 2019-12-09T10:06:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.044+0000 7f26706a2ec0 2 removed watcher, disabling cache 2019-12-09T10:06:00.105 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.103+0000 7f26706a2ec0 20 remove_watcher() i=1 2019-12-09T10:06:00.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.170+0000 7f26706a2ec0 20 remove_watcher() i=2 2019-12-09T10:06:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.225+0000 7f26706a2ec0 20 remove_watcher() i=3 2019-12-09T10:06:00.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.276+0000 7f26706a2ec0 20 remove_watcher() i=4 2019-12-09T10:06:00.314 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.312+0000 7f26706a2ec0 20 remove_watcher() i=5 2019-12-09T10:06:00.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.353+0000 7f26706a2ec0 20 remove_watcher() i=6 2019-12-09T10:06:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.389+0000 7f26706a2ec0 20 remove_watcher() i=7 2019-12-09T10:06:00.399 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:06:00.504 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.502+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.531 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.529+0000 7f49e0e5cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:06:00.531 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.529+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.541 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.539+0000 7f49e0e5cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:06:00.541 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.539+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.549+0000 7f49e0e5cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:06:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.549+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.569+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:06:00.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.738+0000 7f49e0e5cec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:06:00.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.738+0000 7f49e0e5cec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:06:00.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.738+0000 7f49e0e5cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:06:00.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.738+0000 7f49e0e5cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:06:00.741 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.738+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.747+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:06:00.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.747+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.757+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:06:00.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.757+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.768+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:06:00.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.768+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.780+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:06:00.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.781+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.787+0000 7f49e0e5cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:06:00.789 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.787+0000 7f49e0e5cec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:06:00.790 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.787+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.793+0000 7f49e0e5cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:06:00.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.793+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.805+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:06:00.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.805+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.817+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:06:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.817+0000 7f49e0e5cec0 20 zonegroup default 2019-12-09T10:06:00.819 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.817+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.824+0000 7f49e0e5cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:06:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.824+0000 7f49e0e5cec0 10 Cannot find current period zone using local zone 2019-12-09T10:06:00.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.824+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.829+0000 7f49e0e5cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:06:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.829+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.840+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:06:00.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.840+0000 7f49e0e5cec0 20 rados->read ofs=0 len=0 2019-12-09T10:06:00.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.853+0000 7f49e0e5cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:06:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.853+0000 7f49e0e5cec0 20 zone default found 2019-12-09T10:06:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.854+0000 7f49e0e5cec0 20 started sync module instance, tier type = 2019-12-09T10:06:00.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:00.854+0000 7f49e0e5cec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:06:01.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.260+0000 7f49e0e5cec0 20 add_watcher() i=0 2019-12-09T10:06:01.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.260+0000 7f49e0e5cec0 20 add_watcher() i=1 2019-12-09T10:06:01.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.260+0000 7f49e0e5cec0 20 add_watcher() i=2 2019-12-09T10:06:01.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.260+0000 7f49e0e5cec0 20 add_watcher() i=3 2019-12-09T10:06:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.260+0000 7f49e0e5cec0 20 add_watcher() i=4 2019-12-09T10:06:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.260+0000 7f49e0e5cec0 20 add_watcher() i=5 2019-12-09T10:06:01.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.260+0000 7f49e0e5cec0 20 add_watcher() i=6 2019-12-09T10:06:01.292 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.290+0000 7f49e0e5cec0 20 add_watcher() i=7 2019-12-09T10:06:01.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.290+0000 7f49e0e5cec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:06:01.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.291+0000 7f499efdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:06:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.301+0000 7f49ce887700 0 -- 172.21.15.33:0/617002318 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x563c77396a50 msgr2=0x563c773b7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:06:01.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:01.339+0000 7f49ce887700 0 -- 172.21.15.33:0/1667009323 >> [v2:172.21.15.33:6800/9135,v1:172.21.15.33:6801/9135] conn(0x7f4910022480 msgr2=0x7f4910024970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:06:02.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.867+0000 7f490d7fa700 20 reqs_thread_entry: start 2019-12-09T10:06:02.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.867+0000 7f4906ffd700 20 reqs_thread_entry: start 2019-12-09T10:06:02.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.867+0000 7f49e0e5cec0 20 init_complete bucket index max shards: 0 2019-12-09T10:06:02.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.868+0000 7f49057fa700 20 reqs_thread_entry: start 2019-12-09T10:06:02.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.868+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:06:02.915 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.913+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.929+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:02.940 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.938+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:02.970 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:02.968+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:06:03.029 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.027+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.047 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.045+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.058+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.105+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:06:03.175 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.172+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.195+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:03.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.205+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.280 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.278+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:06:03.369 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.367+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.397 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.395+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:03.414 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.412+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.460 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.458+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:06:03.526 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.524+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.545 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.543+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:03.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.553+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.600+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:06:03.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.660+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.678 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.676+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:03.688 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.686+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.734+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:06:03.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.800+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.823+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:03.840 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.838+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.886+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:06:03.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.963+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:03.992+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:04.006 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.004+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.070 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.068+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:06:04.164 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.162+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.191 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.189+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:04.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.207+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.277+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:06:04.386 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.384+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.424+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:04.444 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.442+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.521 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.519+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:06:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.593+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.620+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.635+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.719+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:06:04.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.794+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.823+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.837+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:04.989 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:04.987+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:06:05.041 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.039+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.069 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.067+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:05.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.076+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.120 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.118+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:06:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.187+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.210+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.223+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.263+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:06:05.336 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.334+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.365 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.363+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:05.380 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.378+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.429+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:06:05.506 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.504+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.539 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.537+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.556+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.625+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:06:05.675 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.673+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.690 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.689+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:05.705 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.699+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.738+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:06:05.790 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.788+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.822+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:05.836 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.834+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.872 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.870+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:06:05.919 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.917+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.933+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:05.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.942+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:05.981+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:06:06.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.048+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.072+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:06.083 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.081+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.125+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:06:06.185 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.183+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.205+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.221+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.277+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:06:06.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.349+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.380 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.378+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:06.397 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.395+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.445 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.443+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:06:06.489 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.487+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.505+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.514+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.689 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.687+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:06:06.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.741+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.757+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:06.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.766+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.799 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.797+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:06:06.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.843+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.858+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:06.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.868+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.908 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.907+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:06:06.962 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.960+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:06.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.977+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:06.992 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:06.990+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.040 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.038+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:06:07.094 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.092+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.113 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.111+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:07.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.135+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.189+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:06:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.295+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.331 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.329+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:07.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.346+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.409 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.407+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:06:07.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.472+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.494+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:07.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.513+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.557+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:06:07.653 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.651+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.676 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.674+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:07.690 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.688+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.737+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:06:07.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.799+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.822+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.834+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.875 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.873+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:06:07.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.921+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:07.947 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.945+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:06:07.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:07.958+0000 7f49e0e5cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:06:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.039+0000 7f49e0e5cec0 20 remove_watcher() i=0 2019-12-09T10:06:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.039+0000 7f49e0e5cec0 2 removed watcher, disabling cache 2019-12-09T10:06:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.087+0000 7f49e0e5cec0 20 remove_watcher() i=1 2019-12-09T10:06:08.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.139+0000 7f49e0e5cec0 20 remove_watcher() i=2 2019-12-09T10:06:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.199+0000 7f49e0e5cec0 20 remove_watcher() i=3 2019-12-09T10:06:08.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.254+0000 7f49e0e5cec0 20 remove_watcher() i=4 2019-12-09T10:06:08.299 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.298+0000 7f49e0e5cec0 20 remove_watcher() i=5 2019-12-09T10:06:08.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.347+0000 7f49e0e5cec0 20 remove_watcher() i=6 2019-12-09T10:06:08.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:06:08.382+0000 7f49e0e5cec0 20 remove_watcher() i=7 2019-12-09T10:06:08.390 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2019-12-09T10:06:09.213 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :clean 2019-12-09T10:06:09.910 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :compileJava 2019-12-09T10:06:09.910 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :processResources 2019-12-09T10:06:09.910 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :classes 2019-12-09T10:06:10.211 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:10.211 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :compileTestJava 2019-12-09T10:06:10.211 INFO:teuthology.orchestra.run.smithi033.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T10:06:10.212 INFO:teuthology.orchestra.run.smithi033.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T10:06:10.212 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:10.212 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T10:06:10.212 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :testClasses 2019-12-09T10:06:12.121 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:37:33.582 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.128 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:37:33.590 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.138 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:37:33.599 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.143 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:37:33.627 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.145 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:37:33.607 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.150 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:37:33.634 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.328 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:37:33.789 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.349 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:37:33.810 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.350 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:37:33.834 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:06:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:12.410 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :test 2019-12-09T10:06:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:12.410 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2019-12-09T10:06:13.810 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:13.810 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2019-12-09T10:06:20.609 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:20.610 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2019-12-09T10:06:22.310 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:22.310 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2019-12-09T10:06:28.809 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:28.810 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2019-12-09T10:06:30.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:30.910 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2019-12-09T10:06:37.909 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:37.910 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2019-12-09T10:06:44.710 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:44.710 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2019-12-09T10:06:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:45.910 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2019-12-09T10:06:51.710 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:51.710 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2019-12-09T10:06:57.910 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:57.910 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2019-12-09T10:06:58.410 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:58.410 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2019-12-09T10:06:59.510 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:06:59.510 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2019-12-09T10:07:07.210 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:07:07.210 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2019-12-09T10:07:20.310 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:07:20.310 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2019-12-09T10:07:23.710 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:07:23.710 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2019-12-09T10:07:30.010 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:07:30.010 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2019-12-09T10:07:30.610 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:07:30.610 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2019-12-09T10:07:31.607 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:07:31.607 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------- 2019-12-09T10:07:31.607 INFO:teuthology.orchestra.run.smithi033.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2019-12-09T10:07:31.607 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------- 2019-12-09T10:07:31.608 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:07:31.608 INFO:teuthology.orchestra.run.smithi033.stdout:BUILD SUCCESSFUL in 1m 23s 2019-12-09T10:07:31.608 INFO:teuthology.orchestra.run.smithi033.stdout:5 actionable tasks: 5 executed 2019-12-09T10:07:31.622 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:07:31.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.757+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:31.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.792+0000 7f6a97322ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:31.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.792+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.809+0000 7f6a97322ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:31.812 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.809+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.826+0000 7f6a97322ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:31.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.826+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:31.850 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:31.847+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:32.058 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.055+0000 7f6a97322ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:07:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.055+0000 7f6a97322ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:07:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.055+0000 7f6a97322ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:07:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.055+0000 7f6a97322ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:07:32.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.055+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.080+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:32.083 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.080+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.101+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.101+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.118+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.118+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.153+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.153+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.163+0000 7f6a97322ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:32.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.163+0000 7f6a97322ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:07:32.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.163+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.169+0000 7f6a97322ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:32.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.169+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.185+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:32.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.185+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.208 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.204+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:32.208 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.204+0000 7f6a97322ec0 20 zonegroup default 2019-12-09T10:07:32.208 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.204+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.212+0000 7f6a97322ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:32.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.212+0000 7f6a97322ec0 10 Cannot find current period zone using local zone 2019-12-09T10:07:32.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.213+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.223 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.220+0000 7f6a97322ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:32.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.220+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.230+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:32.233 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.230+0000 7f6a97322ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.245+0000 7f6a97322ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.245+0000 7f6a97322ec0 20 zone default found 2019-12-09T10:07:32.249 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.245+0000 7f6a97322ec0 20 started sync module instance, tier type = 2019-12-09T10:07:32.250 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.245+0000 7f6a97322ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:07:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.590+0000 7f6a97322ec0 20 add_watcher() i=0 2019-12-09T10:07:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.590+0000 7f6a97322ec0 20 add_watcher() i=1 2019-12-09T10:07:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.590+0000 7f6a97322ec0 20 add_watcher() i=2 2019-12-09T10:07:32.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.590+0000 7f6a97322ec0 20 add_watcher() i=3 2019-12-09T10:07:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.590+0000 7f6a97322ec0 20 add_watcher() i=4 2019-12-09T10:07:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.590+0000 7f6a97322ec0 20 add_watcher() i=5 2019-12-09T10:07:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.590+0000 7f6a97322ec0 20 add_watcher() i=6 2019-12-09T10:07:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.625+0000 7f6a97322ec0 20 add_watcher() i=7 2019-12-09T10:07:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.625+0000 7f6a97322ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:07:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.625+0000 7f6a53fef700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:07:32.633 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.630+0000 7f6a84d4d700 0 -- 172.21.15.33:0/2113920042 >> v1:172.21.15.33:6789/0 conn(0x5635e27349c0 legacy=0x5635e2733ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:07:32.676 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:32.673+0000 7f6a83d4b700 0 -- 172.21.15.33:0/1828525774 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x5635e2777dd0 msgr2=0x5635e277a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:07:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.683+0000 7f69c27fc700 20 reqs_thread_entry: start 2019-12-09T10:07:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.683+0000 7f69bbfff700 20 reqs_thread_entry: start 2019-12-09T10:07:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.683+0000 7f6a97322ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:07:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.684+0000 7f69ba7fc700 20 reqs_thread_entry: start 2019-12-09T10:07:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.684+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:07:34.756 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.753+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.777+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:34.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.790+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:34.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.851+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:07:34.897 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.893+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:34.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.911+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:34.923 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.920+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:34.961+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:07:35.019 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.016+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.041 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.038+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.051+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.105+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:07:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.192+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.221+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:35.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.235+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.279+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:07:35.361 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.358+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.379+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.392+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.456+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:07:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.536+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.566+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:35.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.585+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.656 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.653+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:07:35.734 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.731+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.767 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.761+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:35.781 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.778+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.828 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.825+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:07:35.898 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.895+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:35.919 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.915+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:35.928 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:35.925+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.012 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.008+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:07:36.066 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.063+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.087+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:36.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.097+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.157 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.154+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:07:36.242 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.239+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.267+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.291+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.338+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:07:36.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.400+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.421 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.417+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:36.435 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.430+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.481 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.478+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:07:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.552+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.581+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:36.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.597+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.640+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:07:36.697 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.694+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.719 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.715+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:36.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.730+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.792 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.789+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:07:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.832+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.857+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:36.872 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.869+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:36.923+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:07:37.012 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.009+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.122 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.120+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:37.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.139+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.194+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:07:37.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.263+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.287+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:37.302 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.300+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.358+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:07:37.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.439+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.471 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.468+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:37.485 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.482+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.552+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:07:37.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.605+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.623+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:37.635 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.633+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.691 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.689+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:07:37.784 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.781+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.820 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.818+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.833+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:37.886 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.884+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:07:37.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:37.969+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.006 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.004+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:38.026 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.024+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.087 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.085+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:07:38.147 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.145+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.167 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.165+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:38.178 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.176+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.219+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:07:38.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.263+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.279+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:38.290 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.288+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.327 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.325+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:07:38.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.376+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.396 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.394+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:38.406 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.404+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.445+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:07:38.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.516+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.542 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.540+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.554+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.599+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:07:38.653 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.651+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.670 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.668+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:38.679 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.677+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.733+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:07:38.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.796+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.821+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:38.842 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.840+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:38.896 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.894+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:07:38.987 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:38.985+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.023 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.021+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:39.040 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.037+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.098 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.096+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:07:39.162 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.160+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.184 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.182+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:39.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.198+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.250 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.247+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:07:39.311 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.309+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.331 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.328+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:39.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.341+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.381 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.379+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:07:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.424+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.446+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:39.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.455+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.514+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:07:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.604+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.651 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.648+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:39.676 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.674+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.728 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.726+0000 7f6a97322ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:07:39.805 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.802+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.831 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.828+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.844+0000 7f6a97322ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:39.932 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.930+0000 7f6a97322ec0 20 remove_watcher() i=0 2019-12-09T10:07:39.933 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.930+0000 7f6a97322ec0 2 removed watcher, disabling cache 2019-12-09T10:07:39.990 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:39.988+0000 7f6a97322ec0 20 remove_watcher() i=1 2019-12-09T10:07:40.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.035+0000 7f6a97322ec0 20 remove_watcher() i=2 2019-12-09T10:07:40.099 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.097+0000 7f6a97322ec0 20 remove_watcher() i=3 2019-12-09T10:07:40.171 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.169+0000 7f6a97322ec0 20 remove_watcher() i=4 2019-12-09T10:07:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.220+0000 7f6a97322ec0 20 remove_watcher() i=5 2019-12-09T10:07:40.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.273+0000 7f6a97322ec0 20 remove_watcher() i=6 2019-12-09T10:07:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.315+0000 7f6a97322ec0 20 remove_watcher() i=7 2019-12-09T10:07:40.325 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:07:40.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.447+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.480+0000 7f2d1add1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:40.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.480+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.489+0000 7f2d1add1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:40.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.489+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.501 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.499+0000 7f2d1add1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:40.501 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.499+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.520+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.748+0000 7f2d1add1ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:07:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.748+0000 7f2d1add1ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:07:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.748+0000 7f2d1add1ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:07:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.748+0000 7f2d1add1ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:07:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.748+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.761 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.759+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:40.762 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.759+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.774+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:40.777 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.775+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.787+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.787+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.806 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.804+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:40.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.804+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.813+0000 7f2d1add1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.813+0000 7f2d1add1ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:07:40.816 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.813+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.821+0000 7f2d1add1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:40.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.821+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.846+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:40.848 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.846+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.857+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:40.859 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.857+0000 7f2d1add1ec0 20 zonegroup default 2019-12-09T10:07:40.860 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.857+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.865 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.863+0000 7f2d1add1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:40.865 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.863+0000 7f2d1add1ec0 10 Cannot find current period zone using local zone 2019-12-09T10:07:40.865 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.863+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.867+0000 7f2d1add1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:40.869 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.867+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.886+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:40.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.886+0000 7f2d1add1ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:40.899 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.897+0000 7f2d1add1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:40.900 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.897+0000 7f2d1add1ec0 20 zone default found 2019-12-09T10:07:40.900 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.897+0000 7f2d1add1ec0 20 started sync module instance, tier type = 2019-12-09T10:07:40.900 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:40.897+0000 7f2d1add1ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:07:41.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.340+0000 7f2d1add1ec0 20 add_watcher() i=0 2019-12-09T10:07:41.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.340+0000 7f2d1add1ec0 20 add_watcher() i=1 2019-12-09T10:07:41.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.340+0000 7f2d1add1ec0 20 add_watcher() i=2 2019-12-09T10:07:41.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.340+0000 7f2d1add1ec0 20 add_watcher() i=3 2019-12-09T10:07:41.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.340+0000 7f2d1add1ec0 20 add_watcher() i=4 2019-12-09T10:07:41.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.340+0000 7f2d1add1ec0 20 add_watcher() i=5 2019-12-09T10:07:41.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.340+0000 7f2d1add1ec0 20 add_watcher() i=6 2019-12-09T10:07:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.381+0000 7f2d1add1ec0 20 add_watcher() i=7 2019-12-09T10:07:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.381+0000 7f2d1add1ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:07:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.382+0000 7f2c5bfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:07:41.390 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.388+0000 7f2d087fc700 0 -- 172.21.15.33:0/306981779 >> v1:172.21.15.33:6789/0 conn(0x55d41313ec90 legacy=0x55d413151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:07:41.446 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:41.444+0000 7f2d077fa700 0 -- 172.21.15.33:0/3338249701 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x55d413186bb0 msgr2=0x55d413188fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:07:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:42.976+0000 7f2c467fc700 20 reqs_thread_entry: start 2019-12-09T10:07:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:42.977+0000 7f2c3ffff700 20 reqs_thread_entry: start 2019-12-09T10:07:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:42.977+0000 7f2d1add1ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:07:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:42.978+0000 7f2c3e7fc700 20 reqs_thread_entry: start 2019-12-09T10:07:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:42.978+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:07:43.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.056+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.078+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.088+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.155 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.153+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:07:43.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.212+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.235+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:43.248 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.246+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.293+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:07:43.366 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.364+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.394 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.392+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:43.408 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.405+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.461 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.459+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:07:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.519+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.544+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:43.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.555+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.607+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:07:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.688+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.725+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.746+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.805 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.802+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:07:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.863+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.881+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:43.899 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.896+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.932+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:07:43.994 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:43.992+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.013 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.011+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.023+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.067 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.065+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:07:44.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.112+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.132 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.130+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.140+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.181 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.179+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:07:44.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.256+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.279+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.292 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.289+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.362+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:07:44.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.440+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.470 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.467+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.481 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.479+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.545+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:07:44.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.586+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.602+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.610+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.653 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.651+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:07:44.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.696+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.713+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.725 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.723+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.756+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:07:44.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.801+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.822+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.832+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.889 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.886+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:07:44.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.922+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.941 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.939+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:44.950 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.948+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:44.984 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:44.982+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:07:45.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.035+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.051+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:45.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.060+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.098+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:07:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.160+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.179+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:45.191 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.189+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.233+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:07:45.289 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.287+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.304+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:45.319 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.317+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.370+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:07:45.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.446+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.485 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.482+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:45.499 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.495+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.718 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.716+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:07:45.821 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.819+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.852 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.850+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:45.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.875+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:45.939 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:45.937+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:07:46.022 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.020+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.047 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.044+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:46.066 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.064+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.102+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:07:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.172+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.194 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.192+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:46.204 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.202+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.242+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:07:46.306 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.304+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.328 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.325+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.334+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.375+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:07:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.455+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.481+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:46.494 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.492+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.552+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:07:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.624+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.648+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:46.660 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.658+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.713+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:07:46.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.860+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.893+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:46.909 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.906+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:46.951 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.949+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:07:47.001 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:46.999+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.020 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.018+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:47.033 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.030+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.077 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.075+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:07:47.134 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.132+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.155 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.152+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:47.172 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.170+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.229 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.226+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:07:47.269 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.267+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.286 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.283+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:47.296 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.292+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.328 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.326+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:07:47.368 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.366+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.383+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.392+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.436+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:07:47.521 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.519+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.559+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:47.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.571+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.660 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.657+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:07:47.726 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.724+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.745 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.743+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:47.756 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.754+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.800+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:07:47.848 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.846+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:47.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.862+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:47.870+0000 7f2d1add1ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:48.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.002+0000 7f2d1add1ec0 20 remove_watcher() i=0 2019-12-09T10:07:48.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.002+0000 7f2d1add1ec0 2 removed watcher, disabling cache 2019-12-09T10:07:48.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.073+0000 7f2d1add1ec0 20 remove_watcher() i=1 2019-12-09T10:07:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.147+0000 7f2d1add1ec0 20 remove_watcher() i=2 2019-12-09T10:07:48.207 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.205+0000 7f2d1add1ec0 20 remove_watcher() i=3 2019-12-09T10:07:48.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.255+0000 7f2d1add1ec0 20 remove_watcher() i=4 2019-12-09T10:07:48.300 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.298+0000 7f2d1add1ec0 20 remove_watcher() i=5 2019-12-09T10:07:48.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.355+0000 7f2d1add1ec0 20 remove_watcher() i=6 2019-12-09T10:07:48.408 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.406+0000 7f2d1add1ec0 20 remove_watcher() i=7 2019-12-09T10:07:48.416 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:07:48.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.533+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.567+0000 7f05b847fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.568+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.577+0000 7f05b847fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:48.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.577+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.586+0000 7f05b847fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.586+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.609+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.796+0000 7f05b847fec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:07:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.796+0000 7f05b847fec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:07:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.796+0000 7f05b847fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:07:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.796+0000 7f05b847fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:07:48.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.796+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.811+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:48.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.811+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.820+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.820+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.829+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.829+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.844+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:48.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.844+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.852+0000 7f05b847fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.852+0000 7f05b847fec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:07:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.852+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.859+0000 7f05b847fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:48.861 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.859+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.869+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:48.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.869+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.879+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.879+0000 7f05b847fec0 20 zonegroup default 2019-12-09T10:07:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.879+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.886+0000 7f05b847fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:48.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.886+0000 7f05b847fec0 10 Cannot find current period zone using local zone 2019-12-09T10:07:48.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.886+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.893+0000 7f05b847fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:48.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.893+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.903 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.900+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:48.903 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.900+0000 7f05b847fec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.911+0000 7f05b847fec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.911+0000 7f05b847fec0 20 zone default found 2019-12-09T10:07:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.911+0000 7f05b847fec0 20 started sync module instance, tier type = 2019-12-09T10:07:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:48.911+0000 7f05b847fec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:07:49.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.228+0000 7f05b847fec0 20 add_watcher() i=0 2019-12-09T10:07:49.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.228+0000 7f05b847fec0 20 add_watcher() i=1 2019-12-09T10:07:49.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.228+0000 7f05b847fec0 20 add_watcher() i=2 2019-12-09T10:07:49.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.228+0000 7f05b847fec0 20 add_watcher() i=3 2019-12-09T10:07:49.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.228+0000 7f05b847fec0 20 add_watcher() i=4 2019-12-09T10:07:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.228+0000 7f05b847fec0 20 add_watcher() i=5 2019-12-09T10:07:49.232 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.228+0000 7f05b847fec0 20 add_watcher() i=6 2019-12-09T10:07:49.262 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.260+0000 7f05b847fec0 20 add_watcher() i=7 2019-12-09T10:07:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.260+0000 7f05b847fec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:07:49.263 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.260+0000 7f0575fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:07:49.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.263+0000 7f05a5eaa700 0 -- 172.21.15.33:0/3213567568 >> v1:172.21.15.33:6789/0 conn(0x55a34698f0a0 legacy=0x55a34698e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:07:49.304 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:49.302+0000 7f059ffff700 0 -- 172.21.15.33:0/3370060633 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x55a3469c8b70 msgr2=0x55a3469caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:07:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.112+0000 7f04e7fff700 20 reqs_thread_entry: start 2019-12-09T10:07:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.112+0000 7f05b847fec0 20 init_complete bucket index max shards: 0 2019-12-09T10:07:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.112+0000 7f04e5ffb700 20 reqs_thread_entry: start 2019-12-09T10:07:51.116 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.113+0000 7f04cffff700 20 reqs_thread_entry: start 2019-12-09T10:07:51.116 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.113+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:07:51.199 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.197+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.221+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:51.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.233+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.290+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:07:51.365 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.363+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.396+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:51.418 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.416+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.460 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.458+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:07:51.521 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.519+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.541+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:51.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.559+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.607+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:07:51.695 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.693+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.719 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.717+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:51.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.734+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.780+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:07:51.860 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.856+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.889+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:51.908 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.906+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:51.978 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:51.975+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:07:52.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.054+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.085 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.083+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.098+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.148 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.145+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:07:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.224+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.267 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.265+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:52.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.291+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.364 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.362+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:07:52.414 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.412+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.431 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.429+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:52.439 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.437+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.480+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:07:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.550+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.575+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:52.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.588+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.626+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:07:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.680+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.701+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:52.711 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.709+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.752 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.749+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:07:52.806 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.804+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.822+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.833+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.880 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.877+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:07:52.932 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.930+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:52.958 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.956+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:52.970 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:52.968+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.035 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.033+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:07:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.105+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.129 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.127+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:53.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.140+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.180+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:07:53.231 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.229+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.249 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.247+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:53.259 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.257+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.296+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:07:53.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.364+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.398 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.396+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.406+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.451+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:07:53.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.553+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.587+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:53.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.604+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.665 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.663+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:07:53.716 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.713+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.734 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.732+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.742+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.786 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.784+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:07:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.843+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.861+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:53.875 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.873+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.919 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.917+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:07:53.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.969+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:53.990 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:53.988+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:54.003 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.001+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.039 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.037+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:07:54.111 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.109+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.132+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.145+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.198+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:07:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.271+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.299+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.312+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.369+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:07:54.430 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.427+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.447+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:54.462 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.459+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.505 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.503+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:07:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.563+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.591+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:54.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.603+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.669 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.667+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:07:54.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.766+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.792 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.790+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:54.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.800+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.851+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:07:54.897 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.895+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.915 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.913+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.923+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:54.963+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:07:55.023 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.021+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.044 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.042+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:55.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.060+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.108 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.106+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:07:55.159 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.157+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.199+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:55.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.213+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.255+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.310+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.331 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.329+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:55.342 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.339+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.385 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.383+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:07:55.472 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.470+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.499 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.497+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:55.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.512+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.574+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:07:55.652 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.650+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.680+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:55.699 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.697+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.756 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.753+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:07:55.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.807+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.834 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.832+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:55.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.846+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.889 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.887+0000 7f05b847fec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:07:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.931+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.950+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:55.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:55.959+0000 7f05b847fec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.035+0000 7f05b847fec0 20 remove_watcher() i=0 2019-12-09T10:07:56.037 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.035+0000 7f05b847fec0 2 removed watcher, disabling cache 2019-12-09T10:07:56.106 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.104+0000 7f05b847fec0 20 remove_watcher() i=1 2019-12-09T10:07:56.176 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.174+0000 7f05b847fec0 20 remove_watcher() i=2 2019-12-09T10:07:56.243 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.241+0000 7f05b847fec0 20 remove_watcher() i=3 2019-12-09T10:07:56.296 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.294+0000 7f05b847fec0 20 remove_watcher() i=4 2019-12-09T10:07:56.355 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.352+0000 7f05b847fec0 20 remove_watcher() i=5 2019-12-09T10:07:56.404 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.402+0000 7f05b847fec0 20 remove_watcher() i=6 2019-12-09T10:07:56.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.447+0000 7f05b847fec0 20 remove_watcher() i=7 2019-12-09T10:07:56.458 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:07:56.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.571+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.605+0000 7f03a0ab4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:56.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.605+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.616+0000 7f03a0ab4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:56.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.616+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.626+0000 7f03a0ab4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:56.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.626+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.654 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.652+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:56.848 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.846+0000 7f03a0ab4ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:07:56.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.846+0000 7f03a0ab4ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:07:56.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.846+0000 7f03a0ab4ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:07:56.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.846+0000 7f03a0ab4ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:07:56.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.846+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.862+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:56.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.862+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.875+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:56.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.875+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.891 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.889+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:56.891 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.889+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.907 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.905+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:56.908 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.905+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.914 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.912+0000 7f03a0ab4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:56.915 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.912+0000 7f03a0ab4ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:07:56.915 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.912+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.920+0000 7f03a0ab4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:56.922 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.920+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.930+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.930+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.943+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:07:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.944+0000 7f03a0ab4ec0 20 zonegroup default 2019-12-09T10:07:56.946 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.944+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.954+0000 7f03a0ab4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.954+0000 7f03a0ab4ec0 10 Cannot find current period zone using local zone 2019-12-09T10:07:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.954+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.959+0000 7f03a0ab4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:07:56.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.959+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.969+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:07:56.972 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.969+0000 7f03a0ab4ec0 20 rados->read ofs=0 len=0 2019-12-09T10:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.978+0000 7f03a0ab4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.978+0000 7f03a0ab4ec0 20 zone default found 2019-12-09T10:07:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.978+0000 7f03a0ab4ec0 20 started sync module instance, tier type = 2019-12-09T10:07:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:56.978+0000 7f03a0ab4ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:07:57.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.335+0000 7f03a0ab4ec0 20 add_watcher() i=0 2019-12-09T10:07:57.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.336+0000 7f03a0ab4ec0 20 add_watcher() i=1 2019-12-09T10:07:57.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.336+0000 7f03a0ab4ec0 20 add_watcher() i=2 2019-12-09T10:07:57.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.336+0000 7f03a0ab4ec0 20 add_watcher() i=3 2019-12-09T10:07:57.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.336+0000 7f03a0ab4ec0 20 add_watcher() i=4 2019-12-09T10:07:57.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.336+0000 7f03a0ab4ec0 20 add_watcher() i=5 2019-12-09T10:07:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.336+0000 7f03a0ab4ec0 20 add_watcher() i=6 2019-12-09T10:07:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.370+0000 7f03a0ab4ec0 20 add_watcher() i=7 2019-12-09T10:07:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.370+0000 7f03a0ab4ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:07:57.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.370+0000 7f035e7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:07:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.376+0000 7f038e4df700 0 -- 172.21.15.33:0/227556506 >> v2:172.21.15.33:3300/0 conn(0x56333a35e740 msgr2=0x56333a380850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:07:57.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.578+0000 7f038e4df700 0 --2- 172.21.15.33:0/227556506 >> v2:172.21.15.33:3300/0 conn(0x56333a35e740 0x56333a380850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:07:57.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:57.613+0000 7f038dcde700 0 -- 172.21.15.33:0/597216685 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x56333a3765b0 msgr2=0x56333a36ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T10:07:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:58.985+0000 7f02d4ff9700 20 reqs_thread_entry: start 2019-12-09T10:07:58.988 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:58.985+0000 7f02ce7fc700 20 reqs_thread_entry: start 2019-12-09T10:07:58.988 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:58.985+0000 7f03a0ab4ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:07:58.988 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:58.986+0000 7f02ccff9700 20 reqs_thread_entry: start 2019-12-09T10:07:58.988 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:58.986+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:07:59.042 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.039+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.060 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.058+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:59.068 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.065+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.122 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.119+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:07:59.187 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.185+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.207+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:59.222 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.220+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.272 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.270+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:07:59.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.368+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.400+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:59.422 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.420+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.481+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:07:59.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.578+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.601+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:59.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.613+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.672 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.669+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:07:59.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.756+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.788 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.786+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.799+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.858 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.856+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:07:59.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.922+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:07:59.948 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.946+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:07:59.963 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:07:59.961+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.002 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.000+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:08:00.087 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.084+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.108+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.118+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.183 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.181+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:08:00.265 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.263+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.291 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.289+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:00.303 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.301+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.376+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:08:00.446 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.444+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.475+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.491+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.559+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:08:00.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.596+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.612+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:00.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.626+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.662 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.660+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:08:00.732 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.729+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.754 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.752+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.768+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.807+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:08:00.862 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.860+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.883 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.880+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:00.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.893+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.934 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.932+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:08:00.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.977+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:00.998 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:00.996+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.005+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.063+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:08:01.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.124+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.149 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.147+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.160 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.158+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.195 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.193+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:08:01.260 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.258+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.286+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.303 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.301+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.337+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:08:01.391 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.388+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.410 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.407+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.415+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.454+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:08:01.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.506+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.526+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.538 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.536+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.600+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:08:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.657+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.680+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.694+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.767 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.765+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:08:01.827 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.825+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.847 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.845+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.858 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.856+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.916 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.914+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:08:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.963+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:01.984 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.982+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:01.994+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.036 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.034+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:08:02.082 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.080+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.097 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.095+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:02.106 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.104+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.139+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:08:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.198+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.215+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:02.226 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.224+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.256+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:08:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.339+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.361+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:02.373 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.371+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.409+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:08:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.461+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.480+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:02.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.490+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.540 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.538+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:08:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.619+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.645 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.642+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:02.660 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.658+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.733 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.731+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:08:02.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.822+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.849+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:02.868 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.866+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:02.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.922+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:08:02.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:02.977+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.007 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.004+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:03.019 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.017+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.059+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:08:03.118 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.116+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.137+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.148+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.187 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.185+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:08:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.233+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.251+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:03.262 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.260+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.301 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.299+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:08:03.362 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.359+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.387 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.385+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.406+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.462 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.459+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:08:03.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.563+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.590+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:03.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.609+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.648+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:08:03.691 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.689+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.707 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.705+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:08:03.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.713+0000 7f03a0ab4ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:08:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.808+0000 7f03a0ab4ec0 20 remove_watcher() i=0 2019-12-09T10:08:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.808+0000 7f03a0ab4ec0 2 removed watcher, disabling cache 2019-12-09T10:08:03.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.849+0000 7f03a0ab4ec0 20 remove_watcher() i=1 2019-12-09T10:08:03.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.879+0000 7f03a0ab4ec0 20 remove_watcher() i=2 2019-12-09T10:08:03.921 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.919+0000 7f03a0ab4ec0 20 remove_watcher() i=3 2019-12-09T10:08:03.951 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.949+0000 7f03a0ab4ec0 20 remove_watcher() i=4 2019-12-09T10:08:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:03.981+0000 7f03a0ab4ec0 20 remove_watcher() i=5 2019-12-09T10:08:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:04.025+0000 7f03a0ab4ec0 20 remove_watcher() i=6 2019-12-09T10:08:04.061 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:08:04.059+0000 7f03a0ab4ec0 20 remove_watcher() i=7 2019-12-09T10:08:04.071 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2019-12-09T10:08:04.588 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :clean 2019-12-09T10:08:05.185 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :compileJava 2019-12-09T10:08:05.185 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :processResources 2019-12-09T10:08:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :classes 2019-12-09T10:08:05.385 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:05.385 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :compileTestJava 2019-12-09T10:08:05.385 INFO:teuthology.orchestra.run.smithi033.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T10:08:05.386 INFO:teuthology.orchestra.run.smithi033.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T10:08:05.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:05.386 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T10:08:05.386 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :testClasses 2019-12-09T10:08:07.784 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:07.784 INFO:teuthology.orchestra.run.smithi033.stdout:> Task :test 2019-12-09T10:08:07.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:07.785 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2019-12-09T10:08:13.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:13.185 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2019-12-09T10:08:24.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:24.084 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2019-12-09T10:08:28.584 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:28.585 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2019-12-09T10:08:36.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:36.084 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2019-12-09T10:08:39.684 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:39.685 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2019-12-09T10:08:46.284 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:08:46.284 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2019-12-09T10:09:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:13.684 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2019-12-09T10:09:20.984 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:20.984 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2019-12-09T10:09:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:23.885 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2019-12-09T10:09:32.084 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:32.085 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2019-12-09T10:09:36.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:36.185 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2019-12-09T10:09:43.684 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:43.685 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2019-12-09T10:09:43.984 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:43.985 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2019-12-09T10:09:45.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:45.185 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2019-12-09T10:09:48.684 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:48.685 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2019-12-09T10:09:54.584 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:09:54.585 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2019-12-09T10:10:00.685 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:00.685 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2019-12-09T10:10:06.884 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:06.885 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2019-12-09T10:10:09.784 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:09.785 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2019-12-09T10:10:15.985 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:15.985 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2019-12-09T10:10:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:20.885 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2019-12-09T10:10:27.284 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:27.285 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2019-12-09T10:10:31.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:31.785 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2019-12-09T10:10:36.584 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:36.585 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2019-12-09T10:10:39.684 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:39.685 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2019-12-09T10:10:44.884 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:44.885 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2019-12-09T10:10:46.884 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:46.885 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2019-12-09T10:10:54.384 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:54.385 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2019-12-09T10:10:56.285 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:10:56.285 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2019-12-09T10:11:00.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:11:00.885 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2019-12-09T10:11:07.284 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:11:07.285 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2019-12-09T10:11:12.026 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:42:33.487 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.032 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:42:33.493 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.064 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:42:33.525 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.067 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:42:33.527 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.075 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:42:33.558 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.077 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:42:33.560 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.166 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:42:33.626 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:11:12.184 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2019-12-09T10:11:12.223 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:42:33.684 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:12.234 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:42:33.717 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.281 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:42:45.742 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.291 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:42:45.752 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.294 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:42:45.755 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.302 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:42:45.763 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.311 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:42:45.794 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.319 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:42:45.803 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.324 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:42:45.785 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.332 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:42:45.793 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:24.391 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:42:45.875 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:11:27.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:11:27.885 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2019-12-09T10:11:39.685 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:11:39.685 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2019-12-09T10:15:16.285 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:15:16.286 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2019-12-09T10:15:29.285 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:15:29.285 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2019-12-09T10:15:30.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:15:30.886 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2019-12-09T10:15:37.585 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:15:37.586 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2019-12-09T10:15:42.985 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:15:42.986 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2019-12-09T10:15:55.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:15:55.786 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2019-12-09T10:16:10.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:16:10.386 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2019-12-09T10:16:17.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:16:17.786 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2019-12-09T10:16:17.985 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:16:17.986 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2019-12-09T10:16:18.885 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:16:18.886 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2019-12-09T10:16:35.275 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:47:56.735 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.281 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:47:56.740 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.297 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:47:56.780 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.304 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:47:56.787 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.338 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:47:56.798 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.343 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:47:56.803 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.507 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:47:56.966 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.517 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:47:56.999 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:35.529 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:47:56.989 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:16:49.293 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T10:16:49.236+0000 41a65700 -1 handle_data failed with exit code -5 2019-12-09T10:16:49.986 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:16:49.986 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2019-12-09T10:16:56.985 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:16:56.986 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T10:17:27.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:17:27.586 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T10:17:40.285 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:17:40.286 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2019-12-09T10:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:18:06.586 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2019-12-09T10:18:13.986 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:18:13.986 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2019-12-09T10:19:33.490 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:50:54.949 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.496 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:50:54.955 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.545 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:50:55.004 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.546 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:50:55.005 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.569 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:50:55.052 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.572 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:50:55.054 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.634 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:50:55.093 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.659 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:50:55.118 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:19:33.677 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:50:55.159 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:20:22.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:20:22.386 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2019-12-09T10:20:33.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:20:33.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2019-12-09T10:20:58.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:20:58.186 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2019-12-09T10:21:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:21:03.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2019-12-09T10:21:41.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:21:41.287 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2019-12-09T10:21:45.096 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:53:06.554 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.103 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:53:06.561 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.112 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:53:06.571 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.115 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:53:06.597 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.118 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:53:06.577 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.120 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:53:06.602 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.195 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:53:06.654 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.212 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:53:06.671 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:45.225 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:53:06.708 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:21:48.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:21:48.087 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2019-12-09T10:21:54.886 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:21:54.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2019-12-09T10:22:06.886 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:06.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2019-12-09T10:22:07.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:07.088 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2019-12-09T10:22:07.886 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:07.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative STARTED 2019-12-09T10:22:09.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:09.587 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative PASSED 2019-12-09T10:22:12.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:12.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2019-12-09T10:22:14.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:14.187 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2019-12-09T10:22:17.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:17.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2019-12-09T10:22:19.786 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:19.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2019-12-09T10:22:23.486 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:23.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2019-12-09T10:22:24.886 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:24.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2019-12-09T10:22:28.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:28.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2019-12-09T10:22:30.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:30.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2019-12-09T10:22:34.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:34.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2019-12-09T10:22:36.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:36.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2019-12-09T10:22:42.287 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:42.287 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2019-12-09T10:22:46.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:46.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2019-12-09T10:22:56.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:22:56.087 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2019-12-09T10:23:02.587 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:23:02.587 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2019-12-09T10:23:14.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:23:14.187 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2019-12-09T10:23:22.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:23:22.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2019-12-09T10:23:34.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:23:34.187 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2019-12-09T10:23:42.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:23:42.087 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2019-12-09T10:23:52.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:23:52.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2019-12-09T10:23:59.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:23:59.987 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2019-12-09T10:24:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:24:12.587 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2019-12-09T10:24:18.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:24:18.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2019-12-09T10:24:29.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:24:29.987 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2019-12-09T10:24:37.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:24:37.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2019-12-09T10:24:49.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:24:49.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2019-12-09T10:24:56.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:24:56.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2019-12-09T10:25:04.489 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:04.489 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2019-12-09T10:25:07.589 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:07.589 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2019-12-09T10:25:13.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:13.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2019-12-09T10:25:17.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:17.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2019-12-09T10:25:22.287 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:22.287 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2019-12-09T10:25:26.587 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:26.587 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2019-12-09T10:25:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:35.987 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2019-12-09T10:25:43.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:43.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2019-12-09T10:25:56.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:25:56.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2019-12-09T10:26:04.287 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:26:04.287 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2019-12-09T10:26:15.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:26:15.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2019-12-09T10:26:21.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:26:21.987 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2019-12-09T10:26:29.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:26:29.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2019-12-09T10:26:38.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:26:38.388 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2019-12-09T10:26:50.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:26:50.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2019-12-09T10:26:58.478 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:58:19.936 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.486 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:58:19.944 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:26:58.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2019-12-09T10:26:58.495 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:58:19.953 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.498 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:58:19.956 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.508 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:58:19.990 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.514 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:58:19.996 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.683 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:58:20.141 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.690 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:58:20.148 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:26:58.710 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:58:20.192 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:27:11.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2019-12-09T10:27:18.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:27:18.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2019-12-09T10:27:29.287 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:27:29.287 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2019-12-09T10:27:36.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:27:36.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2019-12-09T10:27:48.287 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:27:48.287 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2019-12-09T10:27:55.105 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:59:16.563 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.153 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:59:16.611 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.155 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:59:16.613 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.155 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:59:16.613 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.173 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:59:16.654 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.174 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:59:16.656 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.193 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:00:59:16.651 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.219 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:00:59:16.701 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.222 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:00:59:16.680 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:27:55.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:27:55.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2019-12-09T10:28:04.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:04.987 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2019-12-09T10:28:10.987 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:10.987 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2019-12-09T10:28:22.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:22.188 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2019-12-09T10:28:29.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:29.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2019-12-09T10:28:37.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:37.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2019-12-09T10:28:44.586 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:44.587 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2019-12-09T10:28:53.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:53.087 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2019-12-09T10:28:59.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:28:59.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2019-12-09T10:29:13.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:13.087 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2019-12-09T10:29:16.587 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:16.587 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2019-12-09T10:29:23.589 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:23.589 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2019-12-09T10:29:25.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:25.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2019-12-09T10:29:34.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:34.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2019-12-09T10:29:38.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:38.187 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2019-12-09T10:29:44.493 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:44.493 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2019-12-09T10:29:48.386 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:48.387 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2019-12-09T10:29:55.786 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:29:55.787 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2019-12-09T10:30:04.387 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:04.388 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2019-12-09T10:30:10.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:10.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2019-12-09T10:30:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:11.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2019-12-09T10:30:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:12.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2019-12-09T10:30:18.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:18.187 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2019-12-09T10:30:27.486 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:27.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2019-12-09T10:30:31.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2019-12-09T10:30:38.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:38.686 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2019-12-09T10:30:40.486 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:40.487 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2019-12-09T10:30:44.886 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:44.886 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2019-12-09T10:30:47.988 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:47.988 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2019-12-09T10:30:53.887 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:53.887 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent STARTED 2019-12-09T10:30:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:30:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent PASSED 2019-12-09T10:31:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:31:02.687 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2019-12-09T10:31:12.986 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:31:12.987 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2019-12-09T10:31:20.086 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:31:20.087 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI STARTED 2019-12-09T10:31:30.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:31:30.287 INFO:teuthology.orchestra.run.smithi033.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI PASSED 2019-12-09T10:31:38.486 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:31:38.486 INFO:teuthology.orchestra.run.smithi033.stdout:---------------------------------------------------------------------- 2019-12-09T10:31:38.487 INFO:teuthology.orchestra.run.smithi033.stdout:| Results: SUCCESS (72 tests, 72 successes, 0 failures, 0 skipped) | 2019-12-09T10:31:38.487 INFO:teuthology.orchestra.run.smithi033.stdout:---------------------------------------------------------------------- 2019-12-09T10:31:38.496 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:31:38.496 INFO:teuthology.orchestra.run.smithi033.stdout:BUILD SUCCESSFUL in 23m 34s 2019-12-09T10:31:38.496 INFO:teuthology.orchestra.run.smithi033.stdout:5 actionable tasks: 5 executed 2019-12-09T10:31:38.504 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:31:38.661 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.653+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.684+0000 7fa6947b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:38.689 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.684+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:38.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.692+0000 7fa6947b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:38.697 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.692+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:38.704 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.699+0000 7fa6947b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:38.704 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.699+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:38.727 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.722+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.929+0000 7fa6947b8ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:31:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.929+0000 7fa6947b8ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:31:38.934 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.929+0000 7fa6947b8ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:31:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.929+0000 7fa6947b8ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:31:38.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.929+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.947+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:38.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.947+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.966+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:31:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.966+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:38.987 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.982+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:38.988 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:38.982+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.001+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:31:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.001+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.009+0000 7fa6947b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.009+0000 7fa6947b8ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:31:39.014 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.009+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.020 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.016+0000 7fa6947b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:39.020 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.016+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.036 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.031+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:39.036 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.031+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.046 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.042+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:31:39.047 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.042+0000 7fa6947b8ec0 20 zonegroup default 2019-12-09T10:31:39.047 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.042+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.048+0000 7fa6947b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:39.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.048+0000 7fa6947b8ec0 10 Cannot find current period zone using local zone 2019-12-09T10:31:39.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.048+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.054+0000 7fa6947b8ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:39.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.054+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.071 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.067+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:39.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.067+0000 7fa6947b8ec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:39.083 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.079+0000 7fa6947b8ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:31:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.079+0000 7fa6947b8ec0 20 zone default found 2019-12-09T10:31:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.079+0000 7fa6947b8ec0 20 started sync module instance, tier type = 2019-12-09T10:31:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.079+0000 7fa6947b8ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:31:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.505+0000 7fa6947b8ec0 20 add_watcher() i=0 2019-12-09T10:31:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.505+0000 7fa6947b8ec0 20 add_watcher() i=1 2019-12-09T10:31:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.505+0000 7fa6947b8ec0 20 add_watcher() i=2 2019-12-09T10:31:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.505+0000 7fa6947b8ec0 20 add_watcher() i=3 2019-12-09T10:31:39.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.505+0000 7fa6947b8ec0 20 add_watcher() i=4 2019-12-09T10:31:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.505+0000 7fa6947b8ec0 20 add_watcher() i=5 2019-12-09T10:31:39.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.505+0000 7fa6947b8ec0 20 add_watcher() i=6 2019-12-09T10:31:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.545+0000 7fa6947b8ec0 20 add_watcher() i=7 2019-12-09T10:31:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.545+0000 7fa6947b8ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:31:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.545+0000 7fa6527dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:31:39.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.549+0000 7fa6811e1700 0 -- 172.21.15.33:0/1155025327 >> v2:172.21.15.186:3300/0 conn(0x55cb29106230 msgr2=0x55cb2912e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:31:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:39.593+0000 7fa6819e2700 0 -- 172.21.15.33:0/2184374806 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x55cb2913cbf0 msgr2=0x55cb2913f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:31:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.344+0000 7fa5c8ff9700 20 reqs_thread_entry: start 2019-12-09T10:31:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.344+0000 7fa5c27fc700 20 reqs_thread_entry: start 2019-12-09T10:31:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.344+0000 7fa6947b8ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:31:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.344+0000 7fa5c0ff9700 20 reqs_thread_entry: start 2019-12-09T10:31:41.350 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.344+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:31:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.411+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:41.434 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.429+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:41.450 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1051', time=2019-12-09T12:12:58.230245+0000, chain.objs.size()=9 2019-12-09T10:31:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.1 2019-12-09T10:31:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.1_1 2019-12-09T10:31:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.2 2019-12-09T10:31:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.2_1 2019-12-09T10:31:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.3 2019-12-09T10:31:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.3_1 2019-12-09T10:31:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.4 2019-12-09T10:31:41.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.4_1 2019-12-09T10:31:41.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.445+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~MWLKOoHUb5Dpc5A3WJqyt2jm1NdBKkx.5 2019-12-09T10:31:41.754 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.749+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:41.848 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.844+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:31:41.891 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.886+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:41.911 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.907+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:41.923 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.918+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:41.962+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:31:42.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.023+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.047+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:31:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1073', time=2019-12-09T12:14:30.357207+0000, chain.objs.size()=7 2019-12-09T10:31:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~ptpV_eq5LyGlSeBiMFQ_grs1HBQ-Be-.1 2019-12-09T10:31:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~ptpV_eq5LyGlSeBiMFQ_grs1HBQ-Be-.1_1 2019-12-09T10:31:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~ptpV_eq5LyGlSeBiMFQ_grs1HBQ-Be-.1_2 2019-12-09T10:31:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~ptpV_eq5LyGlSeBiMFQ_grs1HBQ-Be-.2 2019-12-09T10:31:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~ptpV_eq5LyGlSeBiMFQ_grs1HBQ-Be-.2_1 2019-12-09T10:31:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~ptpV_eq5LyGlSeBiMFQ_grs1HBQ-Be-.2_2 2019-12-09T10:31:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~ptpV_eq5LyGlSeBiMFQ_grs1HBQ-Be-.3 2019-12-09T10:31:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1122', time=2019-12-09T12:16:13.000445+0000, chain.objs.size()=9 2019-12-09T10:31:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__multipart_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.1 2019-12-09T10:31:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__shadow_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.1_1 2019-12-09T10:31:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__multipart_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.2 2019-12-09T10:31:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__shadow_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.2_1 2019-12-09T10:31:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.068+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__multipart_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.3 2019-12-09T10:31:42.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.192+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__shadow_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.3_1 2019-12-09T10:31:42.399 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.394+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__multipart_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.4 2019-12-09T10:31:42.399 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.394+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__shadow_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.4_1 2019-12-09T10:31:42.399 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.394+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.71__multipart_key1.2~wEEFo_KI3ZDhRPeDfVGBDkq-8MYp7DT.5 2019-12-09T10:31:42.661 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.655+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:42.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.775+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:31:42.842 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.838+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:42.860 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.855+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1085', time=2019-12-09T12:15:18.186927+0000, chain.objs.size()=7 2019-12-09T10:31:42.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~XE30Eb9AMMfO4gprYN2NqkyzxcNZUQa.1 2019-12-09T10:31:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~XE30Eb9AMMfO4gprYN2NqkyzxcNZUQa.1_1 2019-12-09T10:31:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~XE30Eb9AMMfO4gprYN2NqkyzxcNZUQa.1_2 2019-12-09T10:31:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~XE30Eb9AMMfO4gprYN2NqkyzxcNZUQa.2 2019-12-09T10:31:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~XE30Eb9AMMfO4gprYN2NqkyzxcNZUQa.2_1 2019-12-09T10:31:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~XE30Eb9AMMfO4gprYN2NqkyzxcNZUQa.2_2 2019-12-09T10:31:42.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:42.872+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~XE30Eb9AMMfO4gprYN2NqkyzxcNZUQa.3 2019-12-09T10:31:43.105 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.100+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:43.186 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.182+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:31:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.238+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.260+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:43.286 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.281+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.324+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:31:43.383 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.378+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.398 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.393+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.406+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.406+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.959', time=2019-12-09T12:10:01.912234+0000, chain.objs.size()=2 2019-12-09T10:31:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.406+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.56__shadow_.Jghn3BZcoiHGxba4awx7wOYKhzt7sxU_1 2019-12-09T10:31:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.406+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.56__shadow_.Jghn3BZcoiHGxba4awx7wOYKhzt7sxU_2 2019-12-09T10:31:43.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.518+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:43.646 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.641+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:31:43.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.710+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.730+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.739+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.770+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:31:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.819+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.839 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.835+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.844+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.883 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.878+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:31:43.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.951+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:43.978 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.973+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:43.990 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:43.985+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:44.036 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.031+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:31:44.096 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.091+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:44.118 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.114+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1030', time=2019-12-09T12:11:34.943801+0000, chain.objs.size()=9 2019-12-09T10:31:44.138 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.1 2019-12-09T10:31:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.1_1 2019-12-09T10:31:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.2 2019-12-09T10:31:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.2_1 2019-12-09T10:31:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.3 2019-12-09T10:31:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.3_1 2019-12-09T10:31:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.4 2019-12-09T10:31:44.139 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.4_1 2019-12-09T10:31:44.140 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.133+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.5 2019-12-09T10:31:44.418 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.413+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:44.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.574+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:31:44.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.634+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:44.655 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.650+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:44.664 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.660+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:44.717 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.713+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:31:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.783+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:44.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.802+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1211', time=2019-12-09T12:19:18.310551+0000, chain.objs.size()=9 2019-12-09T10:31:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.1 2019-12-09T10:31:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.1_1 2019-12-09T10:31:44.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.2 2019-12-09T10:31:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.2_1 2019-12-09T10:31:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.3 2019-12-09T10:31:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.3_1 2019-12-09T10:31:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.4 2019-12-09T10:31:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.4_1 2019-12-09T10:31:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:44.818+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~kTA-YTNLpU8pWmRlZWax3Xfce5TNNtW.5 2019-12-09T10:31:45.099 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.093+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:45.195 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.191+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:31:45.239 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.235+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:45.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.251+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:45.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:31:45.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1172', time=2019-12-09T12:17:32.395210+0000, chain.objs.size()=9 2019-12-09T10:31:45.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.1 2019-12-09T10:31:45.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.1_1 2019-12-09T10:31:45.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.2 2019-12-09T10:31:45.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.2_1 2019-12-09T10:31:45.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.3 2019-12-09T10:31:45.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.3_1 2019-12-09T10:31:45.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.4 2019-12-09T10:31:45.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.4_1 2019-12-09T10:31:45.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file2.mpg.2~pHEkHhmi0pA3Vvljl_-Jr9LA4i8lykr.5 2019-12-09T10:31:45.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1223', time=2019-12-09T12:20:03.332268+0000, chain.objs.size()=7 2019-12-09T10:31:45.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.268+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~P3grB-m_vhdCxumeGXgciuS-EQhkVRI.1 2019-12-09T10:31:45.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.269+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~P3grB-m_vhdCxumeGXgciuS-EQhkVRI.1_1 2019-12-09T10:31:45.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.269+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~P3grB-m_vhdCxumeGXgciuS-EQhkVRI.1_2 2019-12-09T10:31:45.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.472+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~P3grB-m_vhdCxumeGXgciuS-EQhkVRI.2 2019-12-09T10:31:45.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.472+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~P3grB-m_vhdCxumeGXgciuS-EQhkVRI.2_1 2019-12-09T10:31:45.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.472+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~P3grB-m_vhdCxumeGXgciuS-EQhkVRI.2_2 2019-12-09T10:31:45.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.615+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~P3grB-m_vhdCxumeGXgciuS-EQhkVRI.3 2019-12-09T10:31:45.806 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.802+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:45.909 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.904+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:31:45.950 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.946+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:45.966 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.962+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1233', time=2019-12-09T12:20:25.628381+0000, chain.objs.size()=7 2019-12-09T10:31:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~l5RKPnLuIZTxr92r5dL4m80HGmPi6Nz.1 2019-12-09T10:31:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~l5RKPnLuIZTxr92r5dL4m80HGmPi6Nz.1_1 2019-12-09T10:31:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~l5RKPnLuIZTxr92r5dL4m80HGmPi6Nz.1_2 2019-12-09T10:31:45.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~l5RKPnLuIZTxr92r5dL4m80HGmPi6Nz.2 2019-12-09T10:31:45.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~l5RKPnLuIZTxr92r5dL4m80HGmPi6Nz.2_1 2019-12-09T10:31:45.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~l5RKPnLuIZTxr92r5dL4m80HGmPi6Nz.2_2 2019-12-09T10:31:45.980 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:45.974+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~l5RKPnLuIZTxr92r5dL4m80HGmPi6Nz.3 2019-12-09T10:31:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.211+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:46.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.319+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:31:46.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.362+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.386+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1894', time=2019-12-09T12:31:14.849316+0000, chain.objs.size()=9 2019-12-09T10:31:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__multipart_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.1 2019-12-09T10:31:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__shadow_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.1_1 2019-12-09T10:31:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__multipart_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.2 2019-12-09T10:31:46.411 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__shadow_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.2_1 2019-12-09T10:31:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__multipart_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.3 2019-12-09T10:31:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__shadow_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.3_1 2019-12-09T10:31:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__multipart_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.4 2019-12-09T10:31:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__shadow_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.4_1 2019-12-09T10:31:46.412 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.405+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.118__multipart_key1.2~TVQn59iGL9otJV11wfIbW8nE99ap_Rv.5 2019-12-09T10:31:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.725+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:46.875 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.870+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:31:46.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.912+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.927+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:46.940 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.936+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:46.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:46.977+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:31:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.042+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:47.071 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.066+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:47.096 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.091+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:47.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.131+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:31:47.181 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.177+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:47.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.196+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:47.216 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.211+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:47.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.211+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1218', time=2019-12-09T12:19:34.128600+0000, chain.objs.size()=9 2019-12-09T10:31:47.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.211+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.1 2019-12-09T10:31:47.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.211+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.1_1 2019-12-09T10:31:47.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.212+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.2 2019-12-09T10:31:47.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.212+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.2_1 2019-12-09T10:31:47.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.212+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.3 2019-12-09T10:31:47.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.212+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.3_1 2019-12-09T10:31:47.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.212+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.4 2019-12-09T10:31:47.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.212+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.4_1 2019-12-09T10:31:47.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.212+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~QOLOtbys3MnX-Uu9gX0C-YzQFEeMsZ4.5 2019-12-09T10:31:47.699 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.695+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:47.795 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.791+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:31:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.867+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.891+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.903+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:47.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:47.956+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:31:48.030 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.026+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:48.057 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.052+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:48.068 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.064+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:48.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.122+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:31:48.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.192+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.219+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.236+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:48.285 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.280+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:31:48.379 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.374+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:48.414 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.410+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:31:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1204', time=2019-12-09T12:18:56.145275+0000, chain.objs.size()=9 2019-12-09T10:31:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.1 2019-12-09T10:31:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.1_1 2019-12-09T10:31:48.455 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.2 2019-12-09T10:31:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.2_1 2019-12-09T10:31:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.3 2019-12-09T10:31:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.3_1 2019-12-09T10:31:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.4 2019-12-09T10:31:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.4_1 2019-12-09T10:31:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~r_m-5YeslJBGPDjsnfGVCMMmacCixBU.5 2019-12-09T10:31:48.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1263', time=2019-12-09T12:21:42.872753+0000, chain.objs.size()=9 2019-12-09T10:31:48.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__multipart_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.1 2019-12-09T10:31:48.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__shadow_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.1_1 2019-12-09T10:31:48.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.450+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__multipart_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.2 2019-12-09T10:31:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.877+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__shadow_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.2_1 2019-12-09T10:31:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.877+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__multipart_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.3 2019-12-09T10:31:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.877+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__shadow_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.3_1 2019-12-09T10:31:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.877+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__multipart_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.4 2019-12-09T10:31:48.882 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.877+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__shadow_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.4_1 2019-12-09T10:31:48.883 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:48.877+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.77__multipart_key1.2~0IouzbXhcnSfxQ8fiwTKpki8J0tetrg.5 2019-12-09T10:31:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.122+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:49.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.230+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:31:49.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.277+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:49.302 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.297+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.311+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.311+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1059', time=2019-12-09T12:13:31.990718+0000, chain.objs.size()=9 2019-12-09T10:31:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.1 2019-12-09T10:31:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.1_1 2019-12-09T10:31:49.317 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.2 2019-12-09T10:31:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.2_1 2019-12-09T10:31:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.3 2019-12-09T10:31:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.3_1 2019-12-09T10:31:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.4 2019-12-09T10:31:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.4_1 2019-12-09T10:31:49.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.312+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~mZNRcbWuUsEY280C0zKo7hOGCfzbB2i.5 2019-12-09T10:31:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.588+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.874+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:31:49.996 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:49.992+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:50.044 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.039+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1228', time=2019-12-09T12:20:21.471945+0000, chain.objs.size()=7 2019-12-09T10:31:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~Wau6o5vlkqRRPMm6s5ZCyBQG2HcVQAW.1 2019-12-09T10:31:50.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~Wau6o5vlkqRRPMm6s5ZCyBQG2HcVQAW.1_1 2019-12-09T10:31:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~Wau6o5vlkqRRPMm6s5ZCyBQG2HcVQAW.1_2 2019-12-09T10:31:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~Wau6o5vlkqRRPMm6s5ZCyBQG2HcVQAW.2 2019-12-09T10:31:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~Wau6o5vlkqRRPMm6s5ZCyBQG2HcVQAW.2_1 2019-12-09T10:31:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__shadow_key1.2~Wau6o5vlkqRRPMm6s5ZCyBQG2HcVQAW.2_2 2019-12-09T10:31:50.065 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.059+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.75__multipart_key1.2~Wau6o5vlkqRRPMm6s5ZCyBQG2HcVQAW.3 2019-12-09T10:31:50.405 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.395+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:50.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.513+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:31:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.556+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:50.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.572+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:50.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:31:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1079', time=2019-12-09T12:15:15.027911+0000, chain.objs.size()=7 2019-12-09T10:31:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-tuBQg9GViw6aJEUOPoxrcFJMMuATww.1 2019-12-09T10:31:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-tuBQg9GViw6aJEUOPoxrcFJMMuATww.1_1 2019-12-09T10:31:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-tuBQg9GViw6aJEUOPoxrcFJMMuATww.1_2 2019-12-09T10:31:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-tuBQg9GViw6aJEUOPoxrcFJMMuATww.2 2019-12-09T10:31:50.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-tuBQg9GViw6aJEUOPoxrcFJMMuATww.2_1 2019-12-09T10:31:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-tuBQg9GViw6aJEUOPoxrcFJMMuATww.2_2 2019-12-09T10:31:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-tuBQg9GViw6aJEUOPoxrcFJMMuATww.3 2019-12-09T10:31:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN', time=2019-12-09T12:22:04.355965+0000, chain.objs.size()=48 2019-12-09T10:31:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.1 2019-12-09T10:31:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.2 2019-12-09T10:31:50.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.3 2019-12-09T10:31:50.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.4 2019-12-09T10:31:50.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.603+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.5 2019-12-09T10:31:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.762+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.6 2019-12-09T10:31:50.767 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.762+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.7 2019-12-09T10:31:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.798+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.8 2019-12-09T10:31:50.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.798+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.9 2019-12-09T10:31:50.894 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.889+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.10 2019-12-09T10:31:50.894 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.889+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.11 2019-12-09T10:31:50.894 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.889+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.OKwCRg_8uQP2Dhijn3buzYHTwlr9fUB.12 2019-12-09T10:31:50.894 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.889+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.13 2019-12-09T10:31:50.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.889+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.uZhg-aIcr3-x7CL2163gGt5er9IJnuU.14 2019-12-09T10:31:50.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.889+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.GW9VbieLQP6_1SSu_j71zoEphnKLzng.15 2019-12-09T10:31:50.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.889+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.mhrKoveYGoP_LakLNVGjj4AhIFzCk-J.16 2019-12-09T10:31:50.899 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:50.895+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.VIkFfS6P-DQdJg2SG8oMdQj-QfIFqt_.17 2019-12-09T10:31:51.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.NZI_WYLYNLw2T7SXlmqttunOGKHU1yY.18 2019-12-09T10:31:51.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.qFOfrcm8T2zAaxWVyuIlasqqc4ti6Hb.19 2019-12-09T10:31:51.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.SrmLP_N6UoMnfLreRgbwwToPkJN5Lgv.20 2019-12-09T10:31:51.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.21 2019-12-09T10:31:51.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.22 2019-12-09T10:31:51.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.23 2019-12-09T10:31:51.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.24 2019-12-09T10:31:51.288 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.282+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.25 2019-12-09T10:31:51.322 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.318+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.26 2019-12-09T10:31:51.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.318+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.27 2019-12-09T10:31:51.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.318+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.28 2019-12-09T10:31:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.601+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.29 2019-12-09T10:31:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.601+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.30 2019-12-09T10:31:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.601+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.31 2019-12-09T10:31:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.601+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.32 2019-12-09T10:31:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.601+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.33 2019-12-09T10:31:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.615+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.34 2019-12-09T10:31:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.615+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.35 2019-12-09T10:31:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.707+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.36 2019-12-09T10:31:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.707+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.37 2019-12-09T10:31:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.707+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.38 2019-12-09T10:31:51.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.708+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.39 2019-12-09T10:31:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.40 2019-12-09T10:31:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.41 2019-12-09T10:31:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.42 2019-12-09T10:31:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.43 2019-12-09T10:31:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.44 2019-12-09T10:31:51.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.45 2019-12-09T10:31:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.46 2019-12-09T10:31:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.47 2019-12-09T10:31:51.874 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:51.868+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.78__multipart_key1.2~_THQM8C0bQuFWIzm4oLiXBwpUtFVWnN.49 2019-12-09T10:31:52.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.099+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:52.268 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.264+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:31:52.321 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.317+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:52.346 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.341+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:52.376 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.370+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:31:52.376 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.370+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1141', time=2019-12-09T12:16:51.602124+0000, chain.objs.size()=9 2019-12-09T10:31:52.376 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.370+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__multipart_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.1 2019-12-09T10:31:52.376 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.370+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__shadow_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.1_1 2019-12-09T10:31:52.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.370+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__multipart_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.2 2019-12-09T10:31:52.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__shadow_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.2_1 2019-12-09T10:31:52.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__multipart_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.3 2019-12-09T10:31:52.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__shadow_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.3_1 2019-12-09T10:31:52.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__multipart_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.4 2019-12-09T10:31:52.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__shadow_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.4_1 2019-12-09T10:31:52.377 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.72__multipart_key1.2~Okf9uBA274DKfAHExBV3eJ0Aln69GJK.5 2019-12-09T10:31:52.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1248', time=2019-12-09T12:20:59.833533+0000, chain.objs.size()=9 2019-12-09T10:31:52.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__multipart_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.1 2019-12-09T10:31:52.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__shadow_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.1_1 2019-12-09T10:31:52.378 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.371+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__multipart_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.2 2019-12-09T10:31:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.545+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__shadow_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.2_1 2019-12-09T10:31:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.592+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__multipart_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.3 2019-12-09T10:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.653+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__shadow_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.3_1 2019-12-09T10:31:52.658 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.653+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__multipart_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.4 2019-12-09T10:31:52.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.698+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__shadow_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.4_1 2019-12-09T10:31:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:52.840+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.76__multipart_key1.2~WU-QPZABqGPVu68bd2ur1Ft1s5R49fN.5 2019-12-09T10:31:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.033+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.243+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:31:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.298+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:53.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.318+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:53.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.333+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:53.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.333+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1043', time=2019-12-09T12:15:24.576881+0000, chain.objs.size()=5 2019-12-09T10:31:53.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.333+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.66__shadow_.sjZBPxJpbOv42qlTmDHFD3p0UMAkfcg_1 2019-12-09T10:31:53.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.333+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.66__shadow_.sjZBPxJpbOv42qlTmDHFD3p0UMAkfcg_2 2019-12-09T10:31:53.338 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.333+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.66__shadow_.sjZBPxJpbOv42qlTmDHFD3p0UMAkfcg_3 2019-12-09T10:31:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.333+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.66__shadow_.sjZBPxJpbOv42qlTmDHFD3p0UMAkfcg_4 2019-12-09T10:31:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.333+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.66__shadow_.sjZBPxJpbOv42qlTmDHFD3p0UMAkfcg_5 2019-12-09T10:31:53.531 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.526+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:53.716 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.711+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:31:53.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.794+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:53.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.821+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.838+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:31:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.838+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob', time=2019-12-09T12:18:10.290410+0000, chain.objs.size()=9 2019-12-09T10:31:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.838+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__multipart_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.7 2019-12-09T10:31:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__shadow_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.7_1 2019-12-09T10:31:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__multipart_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.8 2019-12-09T10:31:53.844 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__shadow_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.8_1 2019-12-09T10:31:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__multipart_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.9 2019-12-09T10:31:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__shadow_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.9_1 2019-12-09T10:31:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__multipart_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.10 2019-12-09T10:31:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__shadow_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.10_1 2019-12-09T10:31:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.74__multipart_key1.2~vKyh9ldtYhZqgyl7fNeu8aUjaLp7Aob.11 2019-12-09T10:31:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1910', time=2019-12-09T12:31:31.832481+0000, chain.objs.size()=9 2019-12-09T10:31:53.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__multipart_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.1 2019-12-09T10:31:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__shadow_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.1_1 2019-12-09T10:31:53.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:53.839+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__multipart_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.2 2019-12-09T10:31:54.021 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.017+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__shadow_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.2_1 2019-12-09T10:31:54.176 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.171+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__multipart_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.3 2019-12-09T10:31:54.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.171+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__shadow_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.3_1 2019-12-09T10:31:54.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.171+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__multipart_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.4 2019-12-09T10:31:54.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.172+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__shadow_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.4_1 2019-12-09T10:31:54.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.172+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.119__multipart_key1/data/file.mpg.2~lAe4gXWYZNtD_XKKcaiVOD9ZQLeqshZ.5 2019-12-09T10:31:54.495 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.490+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.618+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:31:54.693 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.689+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:54.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.708+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T10:31:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1067', time=2019-12-09T12:14:03.092170+0000, chain.objs.size()=9 2019-12-09T10:31:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.1 2019-12-09T10:31:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.1_1 2019-12-09T10:31:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.2 2019-12-09T10:31:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.2_1 2019-12-09T10:31:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.3 2019-12-09T10:31:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.3_1 2019-12-09T10:31:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.4 2019-12-09T10:31:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__shadow_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.4_1 2019-12-09T10:31:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.67__multipart_key1.2~-y8pf082f6ZKBst-C2LRKi4fbfFvG8b.5 2019-12-09T10:31:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1173', time=2019-12-09T12:17:29.473977+0000, chain.objs.size()=9 2019-12-09T10:31:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.1 2019-12-09T10:31:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.1_1 2019-12-09T10:31:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:54.730+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.2 2019-12-09T10:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.122+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.2_1 2019-12-09T10:31:55.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.122+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.3 2019-12-09T10:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.122+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.3_1 2019-12-09T10:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.122+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.4 2019-12-09T10:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.122+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__shadow_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.4_1 2019-12-09T10:31:55.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.122+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.73__multipart_file.mpg.2~Pfv-O4DDO1XWfKGoQqNz51TAJhIr6n9.5 2019-12-09T10:31:55.460 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.456+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:55.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.594+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:31:55.645 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.640+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:55.664 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.660+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:55.675 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.670+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:55.718 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.713+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:31:55.768 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.763+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.780+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:55.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4524.1028', time=2019-12-09T12:11:28.943416+0000, chain.objs.size()=9 2019-12-09T10:31:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.1 2019-12-09T10:31:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.1_1 2019-12-09T10:31:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.2 2019-12-09T10:31:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.2_1 2019-12-09T10:31:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.3 2019-12-09T10:31:55.798 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.3_1 2019-12-09T10:31:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.4 2019-12-09T10:31:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__shadow_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.4_1 2019-12-09T10:31:55.799 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:55.793+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.64__multipart_key1.2~AYIfUQa9BU1dBLXR0wwhJQJTQkxWMiw.5 2019-12-09T10:31:56.114 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.110+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.197+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:31:56.242 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.237+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.253+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T10:31:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2', time=2019-12-09T12:09:12.594233+0000, chain.objs.size()=9 2019-12-09T10:31:56.272 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__multipart_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.1 2019-12-09T10:31:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__shadow_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.1_1 2019-12-09T10:31:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__multipart_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.2 2019-12-09T10:31:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__shadow_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.2_1 2019-12-09T10:31:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__multipart_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.3 2019-12-09T10:31:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__shadow_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.3_1 2019-12-09T10:31:56.273 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__multipart_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.4 2019-12-09T10:31:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__shadow_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.4_1 2019-12-09T10:31:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.267+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.52__multipart_key1.2~HIiPQo4om5Amhr2DeZpGFlRXrfstSc2.5 2019-12-09T10:31:56.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.492+0000 7fa6947b8ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T10:31:56.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.645+0000 7fa6947b8ec0 20 remove_watcher() i=0 2019-12-09T10:31:56.650 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.645+0000 7fa6947b8ec0 2 removed watcher, disabling cache 2019-12-09T10:31:56.707 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.703+0000 7fa6947b8ec0 20 remove_watcher() i=1 2019-12-09T10:31:56.741 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.736+0000 7fa6947b8ec0 20 remove_watcher() i=2 2019-12-09T10:31:56.774 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.769+0000 7fa6947b8ec0 20 remove_watcher() i=3 2019-12-09T10:31:56.823 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.818+0000 7fa6947b8ec0 20 remove_watcher() i=4 2019-12-09T10:31:56.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.872+0000 7fa6947b8ec0 20 remove_watcher() i=5 2019-12-09T10:31:56.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.932+0000 7fa6947b8ec0 20 remove_watcher() i=6 2019-12-09T10:31:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:56.968+0000 7fa6947b8ec0 20 remove_watcher() i=7 2019-12-09T10:31:56.981 INFO:teuthology.orchestra.run.smithi033:> radosgw-admin gc process --include-all 2019-12-09T10:31:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.094+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.123+0000 7fa9f674dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:57.128 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.123+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.133+0000 7fa9f674dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:57.137 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.133+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.139+0000 7fa9f674dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.139+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.161+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.348+0000 7fa9f674dec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:31:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.348+0000 7fa9f674dec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:31:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.348+0000 7fa9f674dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:31:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.348+0000 7fa9f674dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:31:57.353 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.348+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.362+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:57.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.362+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.376+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:31:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.376+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.387+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:57.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.388+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.402+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:31:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.402+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.416 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.412+0000 7fa9f674dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:57.416 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.412+0000 7fa9f674dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:31:57.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.412+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.425 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.420+0000 7fa9f674dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:57.425 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.420+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.433+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.433+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.443+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:31:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.443+0000 7fa9f674dec0 20 zonegroup default 2019-12-09T10:31:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.443+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.451+0000 7fa9f674dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.451+0000 7fa9f674dec0 10 Cannot find current period zone using local zone 2019-12-09T10:31:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.451+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.459+0000 7fa9f674dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:31:57.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.459+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.473 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.469+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:31:57.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.469+0000 7fa9f674dec0 20 rados->read ofs=0 len=0 2019-12-09T10:31:57.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.490+0000 7fa9f674dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:31:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.490+0000 7fa9f674dec0 20 zone default found 2019-12-09T10:31:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.490+0000 7fa9f674dec0 20 started sync module instance, tier type = 2019-12-09T10:31:57.498 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.490+0000 7fa9f674dec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:31:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.804+0000 7fa9f674dec0 20 add_watcher() i=0 2019-12-09T10:31:57.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.804+0000 7fa9f674dec0 20 add_watcher() i=1 2019-12-09T10:31:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.804+0000 7fa9f674dec0 20 add_watcher() i=2 2019-12-09T10:31:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.804+0000 7fa9f674dec0 20 add_watcher() i=3 2019-12-09T10:31:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.804+0000 7fa9f674dec0 20 add_watcher() i=4 2019-12-09T10:31:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.804+0000 7fa9f674dec0 20 add_watcher() i=5 2019-12-09T10:31:57.810 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.806+0000 7fa9f674dec0 20 add_watcher() i=6 2019-12-09T10:31:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.851+0000 7fa9f674dec0 20 add_watcher() i=7 2019-12-09T10:31:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.851+0000 7fa9f674dec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:31:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.851+0000 7fa9b37ee700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:31:57.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.873+0000 7fa9e4178700 0 -- 172.21.15.33:0/329699774 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55af9a8a49c0 msgr2=0x55af9a8a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:31:57.917 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:57.912+0000 7fa9e3176700 0 -- 172.21.15.33:0/3113183423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55af9a8da480 msgr2=0x55af9a8e1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:31:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.346+0000 7fa921ffb700 20 reqs_thread_entry: start 2019-12-09T10:31:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.346+0000 7fa91b7fe700 20 reqs_thread_entry: start 2019-12-09T10:31:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.346+0000 7fa9f674dec0 20 init_complete bucket index max shards: 0 2019-12-09T10:31:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.347+0000 7fa919ffb700 20 reqs_thread_entry: start 2019-12-09T10:31:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.347+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T10:31:59.402 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.397+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:59.433 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.429+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:59.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.438+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:59.489 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.484+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T10:31:59.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.539+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:59.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.555+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.563+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:59.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.632+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T10:31:59.719 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.714+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:59.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.743+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:31:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.758+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:31:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.815+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T10:31:59.991 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:31:59.986+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.023+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.040+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.120 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.116+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T10:32:00.173 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.168+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.198 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.194+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.206+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.272 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.267+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T10:32:00.321 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.316+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.340+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.359+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.398+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T10:32:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.503+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.545 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.540+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:00.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.553+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.596+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T10:32:00.657 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.652+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.669+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.677+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.716 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.711+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T10:32:00.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.755+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.776+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:00.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.788+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.841 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.837+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T10:32:00.929 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.924+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:00.949 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.945+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:00.958 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:00.953+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.000+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T10:32:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.067+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.098 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.093+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.107+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.149 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.145+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T10:32:01.211 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.206+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.234 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.229+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.240+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.281+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T10:32:01.346 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.341+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.371+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:01.400 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.395+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.454+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T10:32:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.496+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.514+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:01.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.523+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.556+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T10:32:01.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.602+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.618+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:01.631 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.627+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.671 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.667+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T10:32:01.717 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.712+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.732 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.728+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:01.741 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.737+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.775 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.770+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T10:32:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.835+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.875 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.871+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:01.891 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.887+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.934 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.930+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T10:32:01.982 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.978+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:01.994+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.007 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.002+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.048+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T10:32:02.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.114+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.145 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.140+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.169+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.231+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T10:32:02.316 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.312+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.340+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.352+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.420 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.415+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T10:32:02.464 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.459+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.478+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.488+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.530 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.521+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T10:32:02.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.578+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.609+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.618+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.663+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T10:32:02.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.709+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.730 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.725+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.740 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.736+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.785 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.780+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T10:32:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.825+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.842+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.855 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.851+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.891 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.886+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T10:32:02.938 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.933+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:02.956 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.951+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:02.965 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:02.960+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.000+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T10:32:03.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.055+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.075+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:03.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.087+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.127 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.122+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T10:32:03.176 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.171+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.194 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.189+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:03.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.198+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.248 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.244+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T10:32:03.314 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.309+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.334 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.329+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:03.349 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.344+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.396+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T10:32:03.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.638+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.662+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:03.680 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.675+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.735 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.731+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T10:32:03.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.788+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.808+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:03.826 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.821+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.884+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T10:32:03.967 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.962+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:03.989+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:04.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.003+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:04.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.060+0000 7fa9f674dec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T10:32:04.135 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.131+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:04.162 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.157+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T10:32:04.177 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.173+0000 7fa9f674dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T10:32:04.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.266+0000 7fa9f674dec0 20 remove_watcher() i=0 2019-12-09T10:32:04.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.266+0000 7fa9f674dec0 2 removed watcher, disabling cache 2019-12-09T10:32:04.310 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.305+0000 7fa9f674dec0 20 remove_watcher() i=1 2019-12-09T10:32:04.350 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.345+0000 7fa9f674dec0 20 remove_watcher() i=2 2019-12-09T10:32:04.403 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.398+0000 7fa9f674dec0 20 remove_watcher() i=3 2019-12-09T10:32:04.442 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.438+0000 7fa9f674dec0 20 remove_watcher() i=4 2019-12-09T10:32:04.501 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.497+0000 7fa9f674dec0 20 remove_watcher() i=5 2019-12-09T10:32:04.535 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.530+0000 7fa9f674dec0 20 remove_watcher() i=6 2019-12-09T10:32:04.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:04.561+0000 7fa9f674dec0 20 remove_watcher() i=7 2019-12-09T10:32:04.573 INFO:teuthology.run_tasks:Running task s3tests... 2019-12-09T10:32:04.589 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True}} 2019-12-09T10:32:04.590 INFO:tasks.s3tests:Downloading s3-tests... 2019-12-09T10:32:04.590 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2019-12-09T10:32:04.590 INFO:teuthology.orchestra.run.smithi033:> git clone -b ceph-master git://git.ceph.com/git/s3-tests.git /home/ubuntu/cephtest/s3-tests 2019-12-09T10:32:04.647 INFO:teuthology.orchestra.run.smithi033.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests'... 2019-12-09T10:32:05.151 INFO:tasks.s3tests:Creating rgw users... 2019-12-09T10:32:05.151 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2019-12-09T10:32:05.152 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --access-key ZHBTIXTHSUXLSWRSDXEF --secret oDJ5U04u4ME4Ug+ussIQJuz1olIG5tmGxthz1TWnJ3XWXzsZzjGWcQ== --email bar.client.0+test@test.test --cluster ceph 2019-12-09T10:32:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.309+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.342+0000 7f08acb6cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:05.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.342+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.356 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.351+0000 7f08acb6cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:05.357 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.351+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.366 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.361+0000 7f08acb6cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:05.367 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.361+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.389 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.384+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.625+0000 7f08acb6cec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:32:05.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.626+0000 7f08acb6cec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:32:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.626+0000 7f08acb6cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:32:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.626+0000 7f08acb6cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:32:05.631 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.626+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.647 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.642+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:05.647 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.642+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.663 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.658+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:05.663 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.658+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.678+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.678+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.697 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.693+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:05.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.693+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.703+0000 7f08acb6cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.703+0000 7f08acb6cec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:32:05.708 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.703+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.714+0000 7f08acb6cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:05.718 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.714+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.734 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.729+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:05.734 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.729+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.746+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:05.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.746+0000 7f08acb6cec0 20 zonegroup default 2019-12-09T10:32:05.751 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.746+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.755+0000 7f08acb6cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:05.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.755+0000 7f08acb6cec0 10 Cannot find current period zone using local zone 2019-12-09T10:32:05.760 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.755+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.771 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.767+0000 7f08acb6cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:05.771 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.767+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.784+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:05.788 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.784+0000 7f08acb6cec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:05.806 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.802+0000 7f08acb6cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:05.806 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.802+0000 7f08acb6cec0 20 zone default found 2019-12-09T10:32:05.806 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.802+0000 7f08acb6cec0 20 started sync module instance, tier type = 2019-12-09T10:32:05.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:05.802+0000 7f08acb6cec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:32:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.209+0000 7f08acb6cec0 20 add_watcher() i=0 2019-12-09T10:32:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.209+0000 7f08acb6cec0 20 add_watcher() i=1 2019-12-09T10:32:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.209+0000 7f08acb6cec0 20 add_watcher() i=2 2019-12-09T10:32:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.209+0000 7f08acb6cec0 20 add_watcher() i=3 2019-12-09T10:32:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.209+0000 7f08acb6cec0 20 add_watcher() i=4 2019-12-09T10:32:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.209+0000 7f08acb6cec0 20 add_watcher() i=5 2019-12-09T10:32:06.215 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.209+0000 7f08acb6cec0 20 add_watcher() i=6 2019-12-09T10:32:06.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.248+0000 7f08acb6cec0 20 add_watcher() i=7 2019-12-09T10:32:06.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.248+0000 7f08acb6cec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:32:06.253 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.248+0000 7f086a7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:32:06.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.254+0000 7f089a597700 0 -- 172.21.15.33:0/3409125743 >> v2:172.21.15.33:3300/0 conn(0x55617b3d57c0 msgr2=0x55617b3d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:32:06.461 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.456+0000 7f089a597700 0 --2- 172.21.15.33:0/3409125743 >> v2:172.21.15.33:3300/0 conn(0x55617b3d57c0 0x55617b3d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:32:06.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:06.503+0000 7f0899d96700 0 -- 172.21.15.33:0/2000781317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55617b3dfdf0 msgr2=0x55617b3e2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:06.970 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:03:28.428 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:06.978 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:03:28.436 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:06.980 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:03:28.462 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:06.984 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:03:28.443 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:06.985 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:03:28.466 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:06.986 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:03:28.445 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:07.129 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:03:28.611 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:07.156 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:03:28.612 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:07.158 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:03:28.616 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:32:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.737+0000 7f07e0ff9700 20 reqs_thread_entry: start 2019-12-09T10:32:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.737+0000 7f07da7fc700 20 reqs_thread_entry: start 2019-12-09T10:32:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.737+0000 7f08acb6cec0 20 init_complete bucket index max shards: 0 2019-12-09T10:32:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.738+0000 7f07d8ff9700 20 reqs_thread_entry: start 2019-12-09T10:32:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.738+0000 7f08acb6cec0 20 get_system_obj_state: rctx=0x7ffc38e66128 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55617b452480 s->prefetch_data=0 2019-12-09T10:32:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.738+0000 7f08acb6cec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T10:32:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.743+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T10:32:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.743+0000 7f08acb6cec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:32:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.743+0000 7f08acb6cec0 20 get_system_obj_state: rctx=0x7ffc38e66138 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x55617b454210 s->prefetch_data=0 2019-12-09T10:32:07.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.743+0000 7f08acb6cec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T10:32:07.756 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.749+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T10:32:07.756 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.749+0000 7f08acb6cec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T10:32:07.756 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.749+0000 7f08acb6cec0 20 get_system_obj_state: rctx=0x7ffc38e66138 obj=default.rgw.meta:users.keys:ZHBTIXTHSUXLSWRSDXEF state=0x55617b454210 s->prefetch_data=0 2019-12-09T10:32:07.756 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.749+0000 7f08acb6cec0 10 cache get: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF : miss 2019-12-09T10:32:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.754+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF info.flags=0x0 2019-12-09T10:32:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.754+0000 7f08acb6cec0 10 adding default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF to cache LRU end 2019-12-09T10:32:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.754+0000 7f08acb6cec0 20 get_system_obj_state: rctx=0x7ffc38e65d48 obj=default.rgw.meta:users.keys:ZHBTIXTHSUXLSWRSDXEF state=0x55617b454210 s->prefetch_data=0 2019-12-09T10:32:07.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.754+0000 7f08acb6cec0 10 cache get: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF : type miss (requested=0x6, cached=0x0) 2019-12-09T10:32:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.759+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF info.flags=0x0 2019-12-09T10:32:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.759+0000 7f08acb6cec0 10 moving default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF to cache LRU end 2019-12-09T10:32:07.764 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.759+0000 7f08acb6cec0 20 get_system_obj_state: rctx=0x7ffc38e65ec8 obj=default.rgw.meta:users.keys:ZHBTIXTHSUXLSWRSDXEF state=0x55617b454210 s->prefetch_data=0 2019-12-09T10:32:07.765 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.759+0000 7f08acb6cec0 10 cache get: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF : type miss (requested=0x6, cached=0x0) 2019-12-09T10:32:07.782 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.764+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF info.flags=0x0 2019-12-09T10:32:07.783 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.764+0000 7f08acb6cec0 10 moving default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF to cache LRU end 2019-12-09T10:32:07.821 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.817+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:32:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.817+0000 7f08acb6cec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:32:07.822 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.817+0000 7f08acb6cec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=617 2019-12-09T10:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.827+0000 7f087affd700 10 RGWWatcher::handle_notify() notify_id 579820585017 cookie 93877167741968 notifier 4619 bl.length()=617 2019-12-09T10:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.827+0000 7f087affd700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:32:07.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:07.827+0000 7f087affd700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:32:08.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.195+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T10:32:08.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.195+0000 7f08acb6cec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T10:32:08.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.195+0000 7f08acb6cec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T10:32:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.208+0000 7f087affd700 10 RGWWatcher::handle_notify() notify_id 579820585018 cookie 93877167741968 notifier 4619 bl.length()=194 2019-12-09T10:32:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.208+0000 7f087affd700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T10:32:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.208+0000 7f087affd700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T10:32:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.507+0000 7f08acb6cec0 10 cache put: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF info.flags=0x7 2019-12-09T10:32:08.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.507+0000 7f08acb6cec0 10 moving default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF to cache LRU end 2019-12-09T10:32:08.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.507+0000 7f08acb6cec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=186 2019-12-09T10:32:08.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.518+0000 7f087affd700 10 RGWWatcher::handle_notify() notify_id 579820585019 cookie 93877167753328 notifier 4619 bl.length()=186 2019-12-09T10:32:08.523 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.518+0000 7f087affd700 10 cache put: name=default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF info.flags=0x7 2019-12-09T10:32:08.523 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.518+0000 7f087affd700 10 moving default.rgw.meta+users.keys+ZHBTIXTHSUXLSWRSDXEF to cache LRU end 2019-12-09T10:32:08.717 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T10:32:08.717 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "bar.client.0", 2019-12-09T10:32:08.717 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Mr. bar.client.0", 2019-12-09T10:32:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T10:32:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T10:32:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T10:32:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [], 2019-12-09T10:32:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [ 2019-12-09T10:32:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T10:32:08.718 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "bar.client.0", 2019-12-09T10:32:08.719 INFO:teuthology.orchestra.run.smithi033.stdout: "access_key": "ZHBTIXTHSUXLSWRSDXEF", 2019-12-09T10:32:08.719 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "oDJ5U04u4ME4Ug+ussIQJuz1olIG5tmGxthz1TWnJ3XWXzsZzjGWcQ==" 2019-12-09T10:32:08.719 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T10:32:08.719 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T10:32:08.719 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [], 2019-12-09T10:32:08.719 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T10:32:08.719 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T10:32:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T10:32:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T10:32:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T10:32:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T10:32:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T10:32:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T10:32:08.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T10:32:08.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T10:32:08.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T10:32:08.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T10:32:08.722 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T10:32:08.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T10:32:08.722 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T10:32:08.722 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T10:32:08.722 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.742+0000 7f08acb6cec0 20 remove_watcher() i=0 2019-12-09T10:32:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.742+0000 7f08acb6cec0 2 removed watcher, disabling cache 2019-12-09T10:32:08.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.788+0000 7f08acb6cec0 20 remove_watcher() i=1 2019-12-09T10:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.838+0000 7f08acb6cec0 20 remove_watcher() i=2 2019-12-09T10:32:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.900+0000 7f08acb6cec0 20 remove_watcher() i=3 2019-12-09T10:32:08.963 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:08.956+0000 7f08acb6cec0 20 remove_watcher() i=4 2019-12-09T10:32:09.014 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.009+0000 7f08acb6cec0 20 remove_watcher() i=5 2019-12-09T10:32:09.081 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.076+0000 7f08acb6cec0 20 remove_watcher() i=6 2019-12-09T10:32:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.112+0000 7f08acb6cec0 20 remove_watcher() i=7 2019-12-09T10:32:09.127 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 9494679254 --totp-seed NZ6K2EXJLTSC4QDEYSE6QOA7AUITBGNTGOPFDAI7RM5DMC45C7QTL662FEWUVFN4 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T10:32:09.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.253+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.283 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.279+0000 7f987ba43ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:09.284 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.279+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.289 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.285+0000 7f987ba43ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:09.289 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.285+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.296 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.291+0000 7f987ba43ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:09.296 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.291+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.311 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.307+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:09.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.477+0000 7f987ba43ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:32:09.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.477+0000 7f987ba43ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:32:09.482 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.477+0000 7f987ba43ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:32:09.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.477+0000 7f987ba43ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:32:09.483 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.477+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.498 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.493+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:09.498 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.493+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.503+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:09.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.503+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.524 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.513+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:09.524 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.513+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.527 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.522+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:09.527 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.522+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.527+0000 7f987ba43ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.527+0000 7f987ba43ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:32:09.532 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.527+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.539+0000 7f987ba43ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.539+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.549+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.549+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.557+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.557+0000 7f987ba43ec0 20 zonegroup default 2019-12-09T10:32:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.557+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.562+0000 7f987ba43ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:09.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.562+0000 7f987ba43ec0 10 Cannot find current period zone using local zone 2019-12-09T10:32:09.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.562+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.572+0000 7f987ba43ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.572+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.582+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:09.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.583+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.597+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:09.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.597+0000 7f987ba43ec0 20 zone default found 2019-12-09T10:32:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.597+0000 7f987ba43ec0 20 started sync module instance, tier type = 2019-12-09T10:32:09.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.597+0000 7f987ba43ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.958+0000 7f987ba43ec0 20 add_watcher() i=0 2019-12-09T10:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.958+0000 7f987ba43ec0 20 add_watcher() i=1 2019-12-09T10:32:09.963 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.958+0000 7f987ba43ec0 20 add_watcher() i=2 2019-12-09T10:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.958+0000 7f987ba43ec0 20 add_watcher() i=3 2019-12-09T10:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.958+0000 7f987ba43ec0 20 add_watcher() i=4 2019-12-09T10:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.958+0000 7f987ba43ec0 20 add_watcher() i=5 2019-12-09T10:32:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.958+0000 7f987ba43ec0 20 add_watcher() i=6 2019-12-09T10:32:09.992 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.987+0000 7f987ba43ec0 20 add_watcher() i=7 2019-12-09T10:32:09.992 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.987+0000 7f987ba43ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:32:09.992 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.987+0000 7f98397e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:32:09.997 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:09.993+0000 7f9868c6d700 0 -- 172.21.15.33:0/3526667778 >> v2:172.21.15.33:3300/0 conn(0x55bcbc52a530 msgr2=0x55bcbc530e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T10:32:10.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:10.195+0000 7f9868c6d700 0 --2- 172.21.15.33:0/3526667778 >> v2:172.21.15.33:3300/0 conn(0x55bcbc52a530 0x55bcbc530e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:32:10.237 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:10.232+0000 7f986846c700 0 -- 172.21.15.33:0/3539290620 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x55bcbc5377b0 msgr2=0x55bcbc539bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:12.077 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.073+0000 7f97ab7fe700 20 reqs_thread_entry: start 2019-12-09T10:32:12.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.073+0000 7f97a97fa700 20 reqs_thread_entry: start 2019-12-09T10:32:12.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.073+0000 7f987ba43ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:32:12.078 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.073+0000 7f97977fe700 20 reqs_thread_entry: start 2019-12-09T10:32:12.079 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.073+0000 7f987ba43ec0 20 get_system_obj_state: rctx=0x7ffedf326958 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55bcbc5a9200 s->prefetch_data=0 2019-12-09T10:32:12.079 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.073+0000 7f987ba43ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T10:32:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.095+0000 7f987ba43ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T10:32:12.099 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.095+0000 7f987ba43ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:32:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.095+0000 7f987ba43ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:32:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.095+0000 7f987ba43ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:32:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.095+0000 7f987ba43ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:32:12.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.095+0000 7f987ba43ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:12.122 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.116+0000 7f987ba43ec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T10:32:12.122 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.116+0000 7f987ba43ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T10:32:12.122 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:12.116+0000 7f987ba43ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:32:14.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.196+0000 7f987ba43ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:32:14.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.196+0000 7f987ba43ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:32:14.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.196+0000 7f987ba43ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=631 2019-12-09T10:32:14.217 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.212+0000 7f985e7fc700 10 RGWWatcher::handle_notify() notify_id 588410519612 cookie 94269101670688 notifier 4968 bl.length()=631 2019-12-09T10:32:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.212+0000 7f985e7fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T10:32:14.218 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.212+0000 7f985e7fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T10:32:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.292+0000 7f987ba43ec0 20 remove_watcher() i=0 2019-12-09T10:32:14.298 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.292+0000 7f987ba43ec0 2 removed watcher, disabling cache 2019-12-09T10:32:14.332 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.326+0000 7f987ba43ec0 20 remove_watcher() i=1 2019-12-09T10:32:14.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.365+0000 7f987ba43ec0 20 remove_watcher() i=2 2019-12-09T10:32:14.409 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.404+0000 7f987ba43ec0 20 remove_watcher() i=3 2019-12-09T10:32:14.749 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:14.743+0000 7f987ba43ec0 20 remove_watcher() i=4 2019-12-09T10:32:15.193 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.188+0000 7f987ba43ec0 20 remove_watcher() i=5 2019-12-09T10:32:15.247 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.241+0000 7f987ba43ec0 20 remove_watcher() i=6 2019-12-09T10:32:15.286 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.280+0000 7f987ba43ec0 20 remove_watcher() i=7 2019-12-09T10:32:15.292 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2019-12-09T10:32:15.293 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key JUOQDBLDEIOWAFNVUCCM --secret pmfRR+VQmuOydmCnksKv+1ettydHfIgVzo9DMlWJQECFIsr9cLDx1Q== --email foo.client.0+test@test.test --cluster ceph 2019-12-09T10:32:15.420 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.415+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.441+0000 7fe798d60ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:15.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.441+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.446+0000 7fe798d60ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:15.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.446+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.451+0000 7fe798d60ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.451+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.467 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.462+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.622+0000 7fe798d60ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:32:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.622+0000 7fe798d60ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:32:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.622+0000 7fe798d60ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:32:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.622+0000 7fe798d60ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:32:15.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.622+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.630+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:15.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.630+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.638+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:15.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.638+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.650+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:15.656 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.650+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.664 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.659+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:15.665 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.659+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.671 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.665+0000 7fe798d60ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:15.671 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.665+0000 7fe798d60ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:32:15.671 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.665+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.669+0000 7fe798d60ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:15.674 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.669+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.677+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:15.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.677+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.690 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.685+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:15.691 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.685+0000 7fe798d60ec0 20 zonegroup default 2019-12-09T10:32:15.691 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.685+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.690+0000 7fe798d60ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.690+0000 7fe798d60ec0 10 Cannot find current period zone using local zone 2019-12-09T10:32:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.690+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.697+0000 7fe798d60ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:15.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.697+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.706+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:15.712 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.706+0000 7fe798d60ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:15.720 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.713+0000 7fe798d60ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:15.720 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.714+0000 7fe798d60ec0 20 zone default found 2019-12-09T10:32:15.721 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.714+0000 7fe798d60ec0 20 started sync module instance, tier type = 2019-12-09T10:32:15.722 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:15.714+0000 7fe798d60ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:32:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.056+0000 7fe798d60ec0 20 add_watcher() i=0 2019-12-09T10:32:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.056+0000 7fe798d60ec0 20 add_watcher() i=1 2019-12-09T10:32:16.062 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.056+0000 7fe798d60ec0 20 add_watcher() i=2 2019-12-09T10:32:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.056+0000 7fe798d60ec0 20 add_watcher() i=3 2019-12-09T10:32:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.056+0000 7fe798d60ec0 20 add_watcher() i=4 2019-12-09T10:32:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.056+0000 7fe798d60ec0 20 add_watcher() i=5 2019-12-09T10:32:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.056+0000 7fe798d60ec0 20 add_watcher() i=6 2019-12-09T10:32:16.100 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.095+0000 7fe798d60ec0 20 add_watcher() i=7 2019-12-09T10:32:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.095+0000 7fe798d60ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:32:16.101 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.095+0000 7fe7567dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:32:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.130+0000 7fe785789700 0 -- >> v2:172.21.15.33:3300/0 conn(0x55eb87f47660 msgr2=0x55eb87f49b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:16.348 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:16.342+0000 7fe78678b700 0 -- 172.21.15.33:0/292506212 >> [v2:172.21.15.186:6810/10132,v1:172.21.15.186:6811/10132] conn(0x55eb87f516f0 msgr2=0x55eb87f51b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:18.261 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.256+0000 7fe6ccff9700 20 reqs_thread_entry: start 2019-12-09T10:32:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.256+0000 7fe6c67fc700 20 reqs_thread_entry: start 2019-12-09T10:32:18.262 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.256+0000 7fe798d60ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:32:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.257+0000 7fe6c4ff9700 20 reqs_thread_entry: start 2019-12-09T10:32:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.257+0000 7fe798d60ec0 20 get_system_obj_state: rctx=0x7ffd66e5cd38 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55eb87fbbed0 s->prefetch_data=0 2019-12-09T10:32:18.263 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.257+0000 7fe798d60ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T10:32:18.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.269+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T10:32:18.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.269+0000 7fe798d60ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:32:18.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.269+0000 7fe798d60ec0 20 get_system_obj_state: rctx=0x7ffd66e5cd48 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x55eb87fbdc60 s->prefetch_data=0 2019-12-09T10:32:18.275 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.269+0000 7fe798d60ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T10:32:18.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.275+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T10:32:18.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.275+0000 7fe798d60ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T10:32:18.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.275+0000 7fe798d60ec0 20 get_system_obj_state: rctx=0x7ffd66e5cd48 obj=default.rgw.meta:users.keys:JUOQDBLDEIOWAFNVUCCM state=0x55eb87fbdc60 s->prefetch_data=0 2019-12-09T10:32:18.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.275+0000 7fe798d60ec0 10 cache get: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM : miss 2019-12-09T10:32:18.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.281+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM info.flags=0x0 2019-12-09T10:32:18.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.281+0000 7fe798d60ec0 10 adding default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM to cache LRU end 2019-12-09T10:32:18.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.281+0000 7fe798d60ec0 20 get_system_obj_state: rctx=0x7ffd66e5c958 obj=default.rgw.meta:users.keys:JUOQDBLDEIOWAFNVUCCM state=0x55eb87fbdc60 s->prefetch_data=0 2019-12-09T10:32:18.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.281+0000 7fe798d60ec0 10 cache get: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM : type miss (requested=0x6, cached=0x0) 2019-12-09T10:32:18.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.287+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM info.flags=0x0 2019-12-09T10:32:18.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.287+0000 7fe798d60ec0 10 moving default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM to cache LRU end 2019-12-09T10:32:18.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.287+0000 7fe798d60ec0 20 get_system_obj_state: rctx=0x7ffd66e5cad8 obj=default.rgw.meta:users.keys:JUOQDBLDEIOWAFNVUCCM state=0x55eb87fbdc60 s->prefetch_data=0 2019-12-09T10:32:18.293 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.287+0000 7fe798d60ec0 10 cache get: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM : type miss (requested=0x6, cached=0x0) 2019-12-09T10:32:18.298 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.292+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM info.flags=0x0 2019-12-09T10:32:18.298 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.292+0000 7fe798d60ec0 10 moving default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM to cache LRU end 2019-12-09T10:32:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.360+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:32:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.360+0000 7fe798d60ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:32:18.366 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.360+0000 7fe798d60ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=617 2019-12-09T10:32:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.373+0000 7fe766ffd700 10 RGWWatcher::handle_notify() notify_id 592705486912 cookie 94470086495008 notifier 4864 bl.length()=617 2019-12-09T10:32:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.373+0000 7fe766ffd700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:32:18.379 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:18.373+0000 7fe766ffd700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:32:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.289+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T10:32:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.289+0000 7fe798d60ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T10:32:19.295 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.289+0000 7fe798d60ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T10:32:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.308+0000 7fe766ffd700 10 RGWWatcher::handle_notify() notify_id 592705486900 cookie 94470086510688 notifier 4864 bl.length()=194 2019-12-09T10:32:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.308+0000 7fe766ffd700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T10:32:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.308+0000 7fe766ffd700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T10:32:19.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.623+0000 7fe798d60ec0 10 cache put: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM info.flags=0x7 2019-12-09T10:32:19.629 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.623+0000 7fe798d60ec0 10 moving default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM to cache LRU end 2019-12-09T10:32:19.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.623+0000 7fe798d60ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=186 2019-12-09T10:32:19.646 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.640+0000 7fe766ffd700 10 RGWWatcher::handle_notify() notify_id 592705486906 cookie 94470086500240 notifier 4864 bl.length()=186 2019-12-09T10:32:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.640+0000 7fe766ffd700 10 cache put: name=default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM info.flags=0x7 2019-12-09T10:32:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:19.640+0000 7fe766ffd700 10 moving default.rgw.meta+users.keys+JUOQDBLDEIOWAFNVUCCM to cache LRU end 2019-12-09T10:32:20.132 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T10:32:20.132 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "foo.client.0", 2019-12-09T10:32:20.132 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Mr. foo.client.0", 2019-12-09T10:32:20.132 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T10:32:20.132 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T10:32:20.133 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T10:32:20.133 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [], 2019-12-09T10:32:20.133 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [ 2019-12-09T10:32:20.133 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T10:32:20.133 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "foo.client.0", 2019-12-09T10:32:20.133 INFO:teuthology.orchestra.run.smithi033.stdout: "access_key": "JUOQDBLDEIOWAFNVUCCM", 2019-12-09T10:32:20.133 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "pmfRR+VQmuOydmCnksKv+1ettydHfIgVzo9DMlWJQECFIsr9cLDx1Q==" 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [], 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T10:32:20.134 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T10:32:20.135 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T10:32:20.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T10:32:20.136 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T10:32:20.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T10:32:20.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T10:32:20.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T10:32:20.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T10:32:20.137 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T10:32:20.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T10:32:20.137 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T10:32:20.137 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T10:32:20.137 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:20.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.173+0000 7fe798d60ec0 20 remove_watcher() i=0 2019-12-09T10:32:20.179 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.173+0000 7fe798d60ec0 2 removed watcher, disabling cache 2019-12-09T10:32:20.276 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.270+0000 7fe798d60ec0 20 remove_watcher() i=1 2019-12-09T10:32:20.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.312+0000 7fe798d60ec0 20 remove_watcher() i=2 2019-12-09T10:32:20.370 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.364+0000 7fe798d60ec0 20 remove_watcher() i=3 2019-12-09T10:32:20.405 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.399+0000 7fe798d60ec0 20 remove_watcher() i=4 2019-12-09T10:32:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.431+0000 7fe798d60ec0 20 remove_watcher() i=5 2019-12-09T10:32:20.481 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.475+0000 7fe798d60ec0 20 remove_watcher() i=6 2019-12-09T10:32:20.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.508+0000 7fe798d60ec0 20 remove_watcher() i=7 2019-12-09T10:32:20.521 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 5663163760 --totp-seed LJUC3F2ODOBZFFMUIVRLIWBYOD2Z2GRK5PQFADJ6PZFW53OWOXVYHIRALMUAJ3F4 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T10:32:20.638 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.632+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.659+0000 7ff9a6b1dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:20.665 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.659+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.665+0000 7ff9a6b1dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:20.671 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.665+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.681 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.672+0000 7ff9a6b1dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:20.682 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.672+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.703 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.698+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:20.947 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.941+0000 7ff9a6b1dec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:32:20.947 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.941+0000 7ff9a6b1dec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:32:20.947 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.941+0000 7ff9a6b1dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:32:20.947 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.941+0000 7ff9a6b1dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:32:20.948 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.941+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.953+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:20.958 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.953+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.963+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:20.969 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.963+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.985 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.980+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:20.986 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.980+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:20.999 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.994+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:21.000 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:20.994+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.002+0000 7ff9a6b1dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.002+0000 7ff9a6b1dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:32:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.003+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.009+0000 7ff9a6b1dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:21.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.009+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.022+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:21.028 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.022+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.043 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.037+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:21.043 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.037+0000 7ff9a6b1dec0 20 zonegroup default 2019-12-09T10:32:21.043 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.037+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.044+0000 7ff9a6b1dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.044+0000 7ff9a6b1dec0 10 Cannot find current period zone using local zone 2019-12-09T10:32:21.050 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.044+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.051+0000 7ff9a6b1dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:21.057 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.051+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.071 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.065+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:21.071 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.065+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:21.090 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.085+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.085+0000 7ff9a6b1dec0 20 zone default found 2019-12-09T10:32:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.085+0000 7ff9a6b1dec0 20 started sync module instance, tier type = 2019-12-09T10:32:21.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.085+0000 7ff9a6b1dec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:32:21.492 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.487+0000 7ff9a6b1dec0 20 add_watcher() i=0 2019-12-09T10:32:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.487+0000 7ff9a6b1dec0 20 add_watcher() i=1 2019-12-09T10:32:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.487+0000 7ff9a6b1dec0 20 add_watcher() i=2 2019-12-09T10:32:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.487+0000 7ff9a6b1dec0 20 add_watcher() i=3 2019-12-09T10:32:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.487+0000 7ff9a6b1dec0 20 add_watcher() i=4 2019-12-09T10:32:21.493 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.487+0000 7ff9a6b1dec0 20 add_watcher() i=5 2019-12-09T10:32:21.494 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.487+0000 7ff9a6b1dec0 20 add_watcher() i=6 2019-12-09T10:32:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.519+0000 7ff9a6b1dec0 20 add_watcher() i=7 2019-12-09T10:32:21.525 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.519+0000 7ff9a6b1dec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:32:21.525 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.519+0000 7ff8e7fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:32:21.529 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.523+0000 7ff994548700 0 -- 172.21.15.33:0/3496080497 >> v2:172.21.15.33:3300/0 conn(0x55ba7aa05f30 msgr2=0x55ba7aa06340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:32:21.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.726+0000 7ff994548700 0 --2- 172.21.15.33:0/3496080497 >> v2:172.21.15.33:3300/0 conn(0x55ba7aa05f30 0x55ba7aa06340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:32:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:21.807+0000 7ff993d47700 0 -- 172.21.15.33:0/2193390546 >> [v2:172.21.15.186:3300/0,v1:172.21.15.186:6789/0] conn(0x55ba7aa0b8b0 msgr2=0x55ba7aa10090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.484+0000 7ff8d27fc700 20 reqs_thread_entry: start 2019-12-09T10:32:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.484+0000 7ff8cbfff700 20 reqs_thread_entry: start 2019-12-09T10:32:23.490 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.484+0000 7ff9a6b1dec0 20 init_complete bucket index max shards: 0 2019-12-09T10:32:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.485+0000 7ff8ca7fc700 20 reqs_thread_entry: start 2019-12-09T10:32:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.485+0000 7ff9a6b1dec0 20 get_system_obj_state: rctx=0x7ffd1fe2e4e8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55ba7aa823b0 s->prefetch_data=0 2019-12-09T10:32:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.485+0000 7ff9a6b1dec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T10:32:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.503+0000 7ff9a6b1dec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T10:32:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.503+0000 7ff9a6b1dec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:32:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.503+0000 7ff9a6b1dec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:32:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.503+0000 7ff9a6b1dec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:32:23.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.503+0000 7ff9a6b1dec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:32:23.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.503+0000 7ff9a6b1dec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:23.527 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.522+0000 7ff9a6b1dec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T10:32:23.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.522+0000 7ff9a6b1dec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T10:32:23.528 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.522+0000 7ff9a6b1dec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:32:23.743 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.737+0000 7ff9a6b1dec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:32:23.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.737+0000 7ff9a6b1dec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:32:23.744 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.737+0000 7ff9a6b1dec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=631 2019-12-09T10:32:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.752+0000 7ff9797fa700 10 RGWWatcher::handle_notify() notify_id 592705486913 cookie 94259409541776 notifier 4995 bl.length()=631 2019-12-09T10:32:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.752+0000 7ff9797fa700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T10:32:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:23.752+0000 7ff9797fa700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T10:32:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.515+0000 7ff9a6b1dec0 20 remove_watcher() i=0 2019-12-09T10:32:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.515+0000 7ff9a6b1dec0 2 removed watcher, disabling cache 2019-12-09T10:32:24.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.590+0000 7ff9a6b1dec0 20 remove_watcher() i=1 2019-12-09T10:32:24.646 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.640+0000 7ff9a6b1dec0 20 remove_watcher() i=2 2019-12-09T10:32:24.691 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.685+0000 7ff9a6b1dec0 20 remove_watcher() i=3 2019-12-09T10:32:24.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.734+0000 7ff9a6b1dec0 20 remove_watcher() i=4 2019-12-09T10:32:24.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.774+0000 7ff9a6b1dec0 20 remove_watcher() i=5 2019-12-09T10:32:24.849 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.844+0000 7ff9a6b1dec0 20 remove_watcher() i=6 2019-12-09T10:32:24.900 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:24.894+0000 7ff9a6b1dec0 20 remove_watcher() i=7 2019-12-09T10:32:24.908 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2019-12-09T10:32:24.908 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --access-key UDTLXQDBXLENGHOOTUQO --secret qpiyXSK+FLPS9ANUK0l5mbK64tBW2xKJJ8nAyN+kAm1Ym7Tl6c3GOA== --email 'testx$tenanteduser.client.0+test@test.test' --cluster ceph 2019-12-09T10:32:25.030 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.025+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.059+0000 7f32e6bb6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:25.064 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.059+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.067+0000 7f32e6bb6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:25.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.067+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.082 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.076+0000 7f32e6bb6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:25.082 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.076+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.099 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.094+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:25.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.253+0000 7f32e6bb6ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:32:25.259 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.253+0000 7f32e6bb6ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:32:25.259 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.253+0000 7f32e6bb6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:32:25.259 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.253+0000 7f32e6bb6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:32:25.259 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.253+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.261+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.261+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.273+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:25.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.273+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.281+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.281+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.305 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.300+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:25.306 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.300+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.311 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.306+0000 7f32e6bb6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:25.312 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.306+0000 7f32e6bb6ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:32:25.312 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.306+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.322 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.316+0000 7f32e6bb6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:25.322 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.316+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.339 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.334+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:25.340 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.334+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.354 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.349+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:25.354 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.349+0000 7f32e6bb6ec0 20 zonegroup default 2019-12-09T10:32:25.354 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.349+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.359 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.354+0000 7f32e6bb6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.354+0000 7f32e6bb6ec0 10 Cannot find current period zone using local zone 2019-12-09T10:32:25.360 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.354+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.361+0000 7f32e6bb6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:25.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.361+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.370+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:25.375 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.370+0000 7f32e6bb6ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.382+0000 7f32e6bb6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.382+0000 7f32e6bb6ec0 20 zone default found 2019-12-09T10:32:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.382+0000 7f32e6bb6ec0 20 started sync module instance, tier type = 2019-12-09T10:32:25.388 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.382+0000 7f32e6bb6ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:32:25.778 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.772+0000 7f32e6bb6ec0 20 add_watcher() i=0 2019-12-09T10:32:25.778 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.772+0000 7f32e6bb6ec0 20 add_watcher() i=1 2019-12-09T10:32:25.778 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.772+0000 7f32e6bb6ec0 20 add_watcher() i=2 2019-12-09T10:32:25.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.772+0000 7f32e6bb6ec0 20 add_watcher() i=3 2019-12-09T10:32:25.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.772+0000 7f32e6bb6ec0 20 add_watcher() i=4 2019-12-09T10:32:25.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.772+0000 7f32e6bb6ec0 20 add_watcher() i=5 2019-12-09T10:32:25.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.772+0000 7f32e6bb6ec0 20 add_watcher() i=6 2019-12-09T10:32:25.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.817+0000 7f32e6bb6ec0 20 add_watcher() i=7 2019-12-09T10:32:25.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.817+0000 7f32e6bb6ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:32:25.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.818+0000 7f3227fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:32:25.828 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.822+0000 7f32d45e1700 0 -- 172.21.15.33:0/1537486167 >> v1:172.21.15.186:6789/0 conn(0x55998e129e90 legacy=0x55998e12d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:25.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:25.871+0000 7f32d35df700 0 -- 172.21.15.33:0/541942906 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x55998e142420 msgr2=0x55998e144840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.790+0000 7f32127fc700 20 reqs_thread_entry: start 2019-12-09T10:32:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.790+0000 7f320bfff700 20 reqs_thread_entry: start 2019-12-09T10:32:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.791+0000 7f32e6bb6ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:32:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.791+0000 7f320a7fc700 20 reqs_thread_entry: start 2019-12-09T10:32:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.791+0000 7f32e6bb6ec0 20 get_system_obj_state: rctx=0x7ffc55c8e9d8 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55998e1a9050 s->prefetch_data=0 2019-12-09T10:32:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.791+0000 7f32e6bb6ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T10:32:27.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.801+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2019-12-09T10:32:27.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.801+0000 7f32e6bb6ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:32:27.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.801+0000 7f32e6bb6ec0 20 get_system_obj_state: rctx=0x7ffc55c8e9e8 obj=default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test state=0x55998e1aae00 s->prefetch_data=0 2019-12-09T10:32:27.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.801+0000 7f32e6bb6ec0 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2019-12-09T10:32:27.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.809+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2019-12-09T10:32:27.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.809+0000 7f32e6bb6ec0 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T10:32:27.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.809+0000 7f32e6bb6ec0 20 get_system_obj_state: rctx=0x7ffc55c8e9e8 obj=default.rgw.meta:users.keys:UDTLXQDBXLENGHOOTUQO state=0x55998e1aae00 s->prefetch_data=0 2019-12-09T10:32:27.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.809+0000 7f32e6bb6ec0 10 cache get: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO : miss 2019-12-09T10:32:27.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.819+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO info.flags=0x0 2019-12-09T10:32:27.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.819+0000 7f32e6bb6ec0 10 adding default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO to cache LRU end 2019-12-09T10:32:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.819+0000 7f32e6bb6ec0 20 get_system_obj_state: rctx=0x7ffc55c8e5f8 obj=default.rgw.meta:users.keys:UDTLXQDBXLENGHOOTUQO state=0x55998e1aae00 s->prefetch_data=0 2019-12-09T10:32:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.819+0000 7f32e6bb6ec0 10 cache get: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO : type miss (requested=0x6, cached=0x0) 2019-12-09T10:32:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.827+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO info.flags=0x0 2019-12-09T10:32:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.827+0000 7f32e6bb6ec0 10 moving default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO to cache LRU end 2019-12-09T10:32:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.827+0000 7f32e6bb6ec0 20 get_system_obj_state: rctx=0x7ffc55c8e778 obj=default.rgw.meta:users.keys:UDTLXQDBXLENGHOOTUQO state=0x55998e1aae00 s->prefetch_data=0 2019-12-09T10:32:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.827+0000 7f32e6bb6ec0 10 cache get: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO : type miss (requested=0x6, cached=0x0) 2019-12-09T10:32:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.835+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO info.flags=0x0 2019-12-09T10:32:27.841 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.835+0000 7f32e6bb6ec0 10 moving default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO to cache LRU end 2019-12-09T10:32:27.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.930+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:32:27.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.930+0000 7f32e6bb6ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:32:27.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.930+0000 7f32e6bb6ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=691 2019-12-09T10:32:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.953+0000 7f32c97fa700 10 RGWWatcher::handle_notify() notify_id 592705486909 cookie 94118001888096 notifier 4637 bl.length()=691 2019-12-09T10:32:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.953+0000 7f32c97fa700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:32:27.959 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:27.953+0000 7f32c97fa700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:32:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.268+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T10:32:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.268+0000 7f32e6bb6ec0 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T10:32:28.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.268+0000 7f32e6bb6ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=224 2019-12-09T10:32:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.284+0000 7f32c97fa700 10 RGWWatcher::handle_notify() notify_id 592705486914 cookie 94118001893312 notifier 4637 bl.length()=224 2019-12-09T10:32:28.290 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.284+0000 7f32c97fa700 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T10:32:28.291 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.284+0000 7f32c97fa700 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T10:32:28.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.529+0000 7f32e6bb6ec0 10 cache put: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO info.flags=0x7 2019-12-09T10:32:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.529+0000 7f32e6bb6ec0 10 moving default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO to cache LRU end 2019-12-09T10:32:28.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.529+0000 7f32e6bb6ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=201 2019-12-09T10:32:28.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.549+0000 7f32c97fa700 10 RGWWatcher::handle_notify() notify_id 592705486910 cookie 94118001878672 notifier 4637 bl.length()=201 2019-12-09T10:32:28.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.549+0000 7f32c97fa700 10 cache put: name=default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO info.flags=0x7 2019-12-09T10:32:28.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.549+0000 7f32c97fa700 10 moving default.rgw.meta+users.keys+UDTLXQDBXLENGHOOTUQO to cache LRU end 2019-12-09T10:32:28.597 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T10:32:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "testx$tenanteduser.client.0", 2019-12-09T10:32:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2019-12-09T10:32:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2019-12-09T10:32:28.597 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T10:32:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T10:32:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [], 2019-12-09T10:32:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [ 2019-12-09T10:32:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T10:32:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "testx$tenanteduser.client.0", 2019-12-09T10:32:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "access_key": "UDTLXQDBXLENGHOOTUQO", 2019-12-09T10:32:28.598 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "qpiyXSK+FLPS9ANUK0l5mbK64tBW2xKJJ8nAyN+kAm1Ym7Tl6c3GOA==" 2019-12-09T10:32:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T10:32:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T10:32:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [], 2019-12-09T10:32:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T10:32:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T10:32:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T10:32:28.599 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T10:32:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T10:32:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T10:32:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T10:32:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T10:32:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T10:32:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T10:32:28.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T10:32:28.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T10:32:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T10:32:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T10:32:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T10:32:28.602 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T10:32:28.602 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:28.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.637+0000 7f32e6bb6ec0 20 remove_watcher() i=0 2019-12-09T10:32:28.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.637+0000 7f32e6bb6ec0 2 removed watcher, disabling cache 2019-12-09T10:32:28.689 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.684+0000 7f32e6bb6ec0 20 remove_watcher() i=1 2019-12-09T10:32:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.733+0000 7f32e6bb6ec0 20 remove_watcher() i=2 2019-12-09T10:32:28.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.801+0000 7f32e6bb6ec0 20 remove_watcher() i=3 2019-12-09T10:32:28.873 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.867+0000 7f32e6bb6ec0 20 remove_watcher() i=4 2019-12-09T10:32:28.940 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.934+0000 7f32e6bb6ec0 20 remove_watcher() i=5 2019-12-09T10:32:28.976 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:28.970+0000 7f32e6bb6ec0 20 remove_watcher() i=6 2019-12-09T10:32:29.010 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.004+0000 7f32e6bb6ec0 20 remove_watcher() i=7 2019-12-09T10:32:29.017 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 1711471369 --totp-seed 5RDCXPX2IKVLT5D3IXOKYQQ7IHIETQ5UPJX2BUFA4IJ7YQ5DD2UP3M2SKUC5CFI4 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T10:32:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.151+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.191 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.186+0000 7ff251939ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:29.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.186+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.196+0000 7ff251939ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:29.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.196+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.212 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.207+0000 7ff251939ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:29.213 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.207+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.232+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.441+0000 7ff251939ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T10:32:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.441+0000 7ff251939ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T10:32:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.441+0000 7ff251939ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T10:32:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.441+0000 7ff251939ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T10:32:29.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.441+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.461 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.456+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.456+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.468+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:29.474 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.468+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.483+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:29.489 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.483+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.498+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.498+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.512+0000 7ff251939ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:29.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.512+0000 7ff251939ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T10:32:29.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.512+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.525 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.519+0000 7ff251939ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:29.525 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.519+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.530+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:29.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.530+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.545+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T10:32:29.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.545+0000 7ff251939ec0 20 zonegroup default 2019-12-09T10:32:29.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.545+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.551+0000 7ff251939ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.551+0000 7ff251939ec0 10 Cannot find current period zone using local zone 2019-12-09T10:32:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.551+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.558+0000 7ff251939ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T10:32:29.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.558+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.567+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T10:32:29.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.567+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.578+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T10:32:29.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.578+0000 7ff251939ec0 20 zone default found 2019-12-09T10:32:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.578+0000 7ff251939ec0 20 started sync module instance, tier type = 2019-12-09T10:32:29.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.578+0000 7ff251939ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T10:32:29.884 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.878+0000 7ff251939ec0 20 add_watcher() i=0 2019-12-09T10:32:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.878+0000 7ff251939ec0 20 add_watcher() i=1 2019-12-09T10:32:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.878+0000 7ff251939ec0 20 add_watcher() i=2 2019-12-09T10:32:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.878+0000 7ff251939ec0 20 add_watcher() i=3 2019-12-09T10:32:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.878+0000 7ff251939ec0 20 add_watcher() i=4 2019-12-09T10:32:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.878+0000 7ff251939ec0 20 add_watcher() i=5 2019-12-09T10:32:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.878+0000 7ff251939ec0 20 add_watcher() i=6 2019-12-09T10:32:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.912+0000 7ff251939ec0 20 add_watcher() i=7 2019-12-09T10:32:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.912+0000 7ff251939ec0 2 all 8 watchers are set, enabling cache 2019-12-09T10:32:29.918 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.912+0000 7ff20f7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T10:32:29.923 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:29.918+0000 7ff23eb63700 0 -- 172.21.15.33:0/1582005073 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x556554c41410 msgr2=0x556554c47cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T10:32:30.126 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:30.120+0000 7ff23eb63700 0 --2- 172.21.15.33:0/1582005073 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x556554c41410 0x556554c47cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T10:32:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:30.182+0000 7ff23f364700 0 -- 172.21.15.33:0/1308042623 >> [v2:172.21.15.33:6800/9135,v1:172.21.15.33:6801/9135] conn(0x7ff180022490 msgr2=0x7ff180024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T10:32:31.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.491+0000 7ff17dffb700 20 reqs_thread_entry: start 2019-12-09T10:32:31.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.491+0000 7ff1777fe700 20 reqs_thread_entry: start 2019-12-09T10:32:31.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.491+0000 7ff251939ec0 20 init_complete bucket index max shards: 0 2019-12-09T10:32:31.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.492+0000 7ff175ffb700 20 reqs_thread_entry: start 2019-12-09T10:32:31.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.492+0000 7ff251939ec0 20 get_system_obj_state: rctx=0x7fff11194c68 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x556554cc2d80 s->prefetch_data=0 2019-12-09T10:32:31.497 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.492+0000 7ff251939ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T10:32:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.505+0000 7ff251939ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T10:32:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.505+0000 7ff251939ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:32:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.505+0000 7ff251939ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T10:32:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.505+0000 7ff251939ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T10:32:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.505+0000 7ff251939ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T10:32:31.510 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.505+0000 7ff251939ec0 20 rados->read ofs=0 len=0 2019-12-09T10:32:31.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.517+0000 7ff251939ec0 20 rados_obj.operate() r=0 bl.length=491 2019-12-09T10:32:31.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.517+0000 7ff251939ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T10:32:31.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.517+0000 7ff251939ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:32:31.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.611+0000 7ff251939ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:32:31.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.611+0000 7ff251939ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:32:31.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.611+0000 7ff251939ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=705 2019-12-09T10:32:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.622+0000 7ff21ffff700 10 RGWWatcher::handle_notify() notify_id 592705486911 cookie 93893702137648 notifier 5025 bl.length()=705 2019-12-09T10:32:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.622+0000 7ff21ffff700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T10:32:31.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:31.622+0000 7ff21ffff700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T10:32:32.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.193+0000 7ff251939ec0 20 remove_watcher() i=0 2019-12-09T10:32:32.198 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.193+0000 7ff251939ec0 2 removed watcher, disabling cache 2019-12-09T10:32:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.291+0000 7ff251939ec0 20 remove_watcher() i=1 2019-12-09T10:32:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.338+0000 7ff251939ec0 20 remove_watcher() i=2 2019-12-09T10:32:32.398 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.393+0000 7ff251939ec0 20 remove_watcher() i=3 2019-12-09T10:32:32.453 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.448+0000 7ff251939ec0 20 remove_watcher() i=4 2019-12-09T10:32:32.519 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.514+0000 7ff251939ec0 20 remove_watcher() i=5 2019-12-09T10:32:32.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.560+0000 7ff251939ec0 20 remove_watcher() i=6 2019-12-09T10:32:32.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T10:32:32.597+0000 7ff251939ec0 20 remove_watcher() i=7 2019-12-09T10:32:32.609 INFO:tasks.s3tests:Configuring s3-tests... 2019-12-09T10:32:32.609 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest/s3-tests && ./bootstrap 2019-12-09T10:32:37.989 INFO:teuthology.orchestra.run.smithi033.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T10:32:38.142 INFO:teuthology.orchestra.run.smithi033.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T10:32:38.180 INFO:teuthology.orchestra.run.smithi033.stdout:Loading mirror speeds from cached hostfile 2019-12-09T10:32:38.180 INFO:teuthology.orchestra.run.smithi033.stdout: * base: mirror.ash.fastserv.com 2019-12-09T10:32:38.181 INFO:teuthology.orchestra.run.smithi033.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T10:32:38.181 INFO:teuthology.orchestra.run.smithi033.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T10:32:38.182 INFO:teuthology.orchestra.run.smithi033.stdout: * updates: mirror.cogentco.com 2019-12-09T10:32:38.857 INFO:teuthology.orchestra.run.smithi033.stdout:7 packages excluded due to repository priority protections 2019-12-09T10:32:39.201 INFO:teuthology.orchestra.run.smithi033.stdout:Resolving Dependencies 2019-12-09T10:32:39.202 INFO:teuthology.orchestra.run.smithi033.stdout:--> Running transaction check 2019-12-09T10:32:39.202 INFO:teuthology.orchestra.run.smithi033.stdout:---> Package libffi-devel.x86_64 0:3.0.13-18.el7 will be installed 2019-12-09T10:32:39.597 INFO:teuthology.orchestra.run.smithi033.stdout:--> Finished Dependency Resolution 2019-12-09T10:32:39.838 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:39.839 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies Resolved 2019-12-09T10:32:39.839 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:39.840 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T10:32:39.840 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2019-12-09T10:32:39.840 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T10:32:39.840 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2019-12-09T10:32:39.840 INFO:teuthology.orchestra.run.smithi033.stdout: libffi-devel x86_64 3.0.13-18.el7 base 23 k 2019-12-09T10:32:39.840 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:39.841 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2019-12-09T10:32:39.841 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2019-12-09T10:32:39.841 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2019-12-09T10:32:39.841 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:39.841 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 23 k 2019-12-09T10:32:39.841 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 27 k 2019-12-09T10:32:39.842 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading packages: 2019-12-09T10:32:39.925 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2019-12-09T10:32:39.928 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2019-12-09T10:32:39.930 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded 2019-12-09T10:32:39.930 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2019-12-09T10:32:40.389 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T10:32:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T10:32:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:40.796 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2019-12-09T10:32:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: libffi-devel.x86_64 0:3.0.13-18.el7 2019-12-09T10:32:40.796 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:40.796 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2019-12-09T10:32:40.951 INFO:teuthology.orchestra.run.smithi033.stdout:New python executable in /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python 2019-12-09T10:32:42.228 INFO:teuthology.orchestra.run.smithi033.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T10:32:42.511 INFO:teuthology.orchestra.run.smithi033.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T10:32:42.652 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting pip 2019-12-09T10:32:42.732 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T10:32:42.799 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: pip 2019-12-09T10:32:42.799 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: pip 9.0.1 2019-12-09T10:32:42.861 INFO:teuthology.orchestra.run.smithi033.stdout: Uninstalling pip-9.0.1: 2019-12-09T10:32:42.905 INFO:teuthology.orchestra.run.smithi033.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T10:32:43.505 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed pip-19.3.1 2019-12-09T10:32:43.779 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T10:32:44.324 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting setuptools==32.3.1 2019-12-09T10:32:44.327 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T10:32:44.369 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: setuptools 2019-12-09T10:32:44.369 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T10:32:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T10:32:44.393 INFO:teuthology.orchestra.run.smithi033.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T10:32:44.527 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed setuptools-32.3.1 2019-12-09T10:32:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T10:32:45.105 INFO:teuthology.orchestra.run.smithi033.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T10:32:45.175 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting nose>=1.0.0 2019-12-09T10:32:45.177 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T10:32:45.317 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting boto>=2.6.0 2019-12-09T10:32:45.326 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T10:32:45.833 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting boto3>=1.0.0 2019-12-09T10:32:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/1f/a5/6b25e39aea40bc03163615c07572d90aaddc9951f5af9c21204d3da46398/boto3-1.10.34-py2.py3-none-any.whl (128kB) 2019-12-09T10:32:46.010 INFO:teuthology.orchestra.run.smithi033.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T10:32:46.277 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting gevent>=1.0 2019-12-09T10:32:46.293 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T10:32:46.557 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting isodate>=0.4.4 2019-12-09T10:32:46.566 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl (45kB) 2019-12-09T10:32:46.703 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting requests>=0.14.0 2019-12-09T10:32:46.713 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) 2019-12-09T10:32:47.024 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting pytz>=2011k 2019-12-09T10:32:47.034 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB) 2019-12-09T10:32:47.252 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting ordereddict 2019-12-09T10:32:47.269 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz 2019-12-09T10:32:47.561 INFO:teuthology.orchestra.run.smithi033.stdout:Processing /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488/httplib2-0.14.0-cp27-none-any.whl 2019-12-09T10:32:47.936 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting lxml 2019-12-09T10:32:47.950 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T10:32:48.136 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting jmespath<1.0.0,>=0.7.1 2019-12-09T10:32:48.145 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl 2019-12-09T10:32:48.776 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting botocore<1.14.0,>=1.13.34 2019-12-09T10:32:48.787 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/59/cb/b4772a4abc128f6fd637af9d7fc93a1db11617859af680fc2b2e6282eb95/botocore-1.13.34-py2.py3-none-any.whl (5.8MB) 2019-12-09T10:32:49.455 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting s3transfer<0.3.0,>=0.2.0 2019-12-09T10:32:49.465 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB) 2019-12-09T10:32:49.671 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T10:32:49.674 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T10:32:49.753 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting six 2019-12-09T10:32:49.763 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T10:32:49.835 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting chardet<3.1.0,>=3.0.2 2019-12-09T10:32:49.846 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 2019-12-09T10:32:49.949 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting idna<2.9,>=2.5 2019-12-09T10:32:49.959 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 2019-12-09T10:32:50.096 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 2019-12-09T10:32:50.106 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB) 2019-12-09T10:32:50.241 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting certifi>=2017.4.17 2019-12-09T10:32:50.251 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB) 2019-12-09T10:32:50.369 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting docutils<0.16,>=0.10 2019-12-09T10:32:50.379 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl (548kB) 2019-12-09T10:32:50.568 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7" 2019-12-09T10:32:50.578 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 2019-12-09T10:32:50.662 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" 2019-12-09T10:32:50.671 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl 2019-12-09T10:32:50.707 INFO:teuthology.orchestra.run.smithi033.stdout:Building wheels for collected packages: ordereddict 2019-12-09T10:32:50.707 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for ordereddict (setup.py): started 2019-12-09T10:32:50.931 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for ordereddict (setup.py): finished with status 'done' 2019-12-09T10:32:50.932 INFO:teuthology.orchestra.run.smithi033.stdout: Created wheel for ordereddict: filename=ordereddict-1.1-cp27-none-any.whl size=3327 sha256=2089017af1b344cf7f26fbff003ef4e77bffbe8003430af70260ad77f075eeab 2019-12-09T10:32:50.932 INFO:teuthology.orchestra.run.smithi033.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 2019-12-09T10:32:50.934 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully built ordereddict 2019-12-09T10:32:51.015 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: PyYAML, nose, boto, jmespath, docutils, six, python-dateutil, urllib3, botocore, futures, s3transfer, boto3, bunch, greenlet, gevent, isodate, chardet, idna, certifi, requests, pytz, ordereddict, httplib2, lxml 2019-12-09T10:32:53.309 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 boto3-1.10.34 botocore-1.13.34 bunch-1.0.1 certifi-2019.11.28 chardet-3.0.4 docutils-0.15.2 futures-3.3.0 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 idna-2.8 isodate-0.6.0 jmespath-0.9.4 lxml-4.4.2 nose-1.3.7 ordereddict-1.1 python-dateutil-2.8.0 pytz-2019.3 requests-2.22.0 s3transfer-0.2.1 six-1.13.0 urllib3-1.25.7 2019-12-09T10:32:53.627 INFO:teuthology.orchestra.run.smithi033.stdout:running develop 2019-12-09T10:32:53.681 INFO:teuthology.orchestra.run.smithi033.stdout:running egg_info 2019-12-09T10:32:53.682 INFO:teuthology.orchestra.run.smithi033.stdout:creating s3tests.egg-info 2019-12-09T10:32:53.682 INFO:teuthology.orchestra.run.smithi033.stdout:writing requirements to s3tests.egg-info/requires.txt 2019-12-09T10:32:53.682 INFO:teuthology.orchestra.run.smithi033.stdout:writing s3tests.egg-info/PKG-INFO 2019-12-09T10:32:53.682 INFO:teuthology.orchestra.run.smithi033.stdout:writing top-level names to s3tests.egg-info/top_level.txt 2019-12-09T10:32:53.682 INFO:teuthology.orchestra.run.smithi033.stdout:writing dependency_links to s3tests.egg-info/dependency_links.txt 2019-12-09T10:32:53.683 INFO:teuthology.orchestra.run.smithi033.stdout:writing entry points to s3tests.egg-info/entry_points.txt 2019-12-09T10:32:53.683 INFO:teuthology.orchestra.run.smithi033.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T10:32:53.685 INFO:teuthology.orchestra.run.smithi033.stdout:reading manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T10:32:53.686 INFO:teuthology.orchestra.run.smithi033.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T10:32:53.687 INFO:teuthology.orchestra.run.smithi033.stdout:running build_ext 2019-12-09T10:32:53.687 INFO:teuthology.orchestra.run.smithi033.stdout:Creating /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/s3tests.egg-link (link to .) 2019-12-09T10:32:53.688 INFO:teuthology.orchestra.run.smithi033.stdout:Adding s3tests 0.0.1 to easy-install.pth file 2019-12-09T10:32:53.690 INFO:teuthology.orchestra.run.smithi033.stdout:Installing s3tests-fuzz-headers script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:32:53.690 INFO:teuthology.orchestra.run.smithi033.stdout:Installing s3tests-generate-objects script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:32:53.691 INFO:teuthology.orchestra.run.smithi033.stdout:Installing s3tests-test-roundtrip script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:32:53.691 INFO:teuthology.orchestra.run.smithi033.stdout:Installing s3tests-analysis-rwstats script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:32:53.691 INFO:teuthology.orchestra.run.smithi033.stdout:Installing s3tests-test-readwrite script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T10:32:53.691 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.691 INFO:teuthology.orchestra.run.smithi033.stdout:Installed /home/ubuntu/cephtest/s3-tests 2019-12-09T10:32:53.692 INFO:teuthology.orchestra.run.smithi033.stdout:Processing dependencies for s3tests==0.0.1 2019-12-09T10:32:53.744 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for isodate==0.6.0 2019-12-09T10:32:53.744 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: isodate 0.6.0 2019-12-09T10:32:53.744 INFO:teuthology.orchestra.run.smithi033.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T10:32:53.744 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.744 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.745 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for gevent==1.4.0 2019-12-09T10:32:53.745 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: gevent 1.4.0 2019-12-09T10:32:53.745 INFO:teuthology.orchestra.run.smithi033.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T10:32:53.745 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.746 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.747 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for bunch==1.0.1 2019-12-09T10:32:53.747 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: bunch 1.0.1 2019-12-09T10:32:53.747 INFO:teuthology.orchestra.run.smithi033.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T10:32:53.747 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.747 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.747 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for PyYAML==5.2 2019-12-09T10:32:53.748 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: PyYAML 5.2 2019-12-09T10:32:53.748 INFO:teuthology.orchestra.run.smithi033.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T10:32:53.748 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.748 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.750 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for boto3==1.10.34 2019-12-09T10:32:53.750 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: boto3 1.10.34 2019-12-09T10:32:53.750 INFO:teuthology.orchestra.run.smithi033.stdout:Adding boto3 1.10.34 to easy-install.pth file 2019-12-09T10:32:53.750 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.750 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.751 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for boto==2.49.0 2019-12-09T10:32:53.751 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: boto 2.49.0 2019-12-09T10:32:53.751 INFO:teuthology.orchestra.run.smithi033.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T10:32:53.751 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.751 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.751 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for six==1.13.0 2019-12-09T10:32:53.752 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: six 1.13.0 2019-12-09T10:32:53.753 INFO:teuthology.orchestra.run.smithi033.stdout:Adding six 1.13.0 to easy-install.pth file 2019-12-09T10:32:53.753 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.753 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.753 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for greenlet==0.4.15 2019-12-09T10:32:53.754 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: greenlet 0.4.15 2019-12-09T10:32:53.754 INFO:teuthology.orchestra.run.smithi033.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T10:32:53.754 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.754 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.754 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for s3transfer==0.2.1 2019-12-09T10:32:53.755 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: s3transfer 0.2.1 2019-12-09T10:32:53.755 INFO:teuthology.orchestra.run.smithi033.stdout:Adding s3transfer 0.2.1 to easy-install.pth file 2019-12-09T10:32:53.755 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.755 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.755 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for botocore==1.13.34 2019-12-09T10:32:53.758 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: botocore 1.13.34 2019-12-09T10:32:53.758 INFO:teuthology.orchestra.run.smithi033.stdout:Adding botocore 1.13.34 to easy-install.pth file 2019-12-09T10:32:53.758 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.758 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.758 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for jmespath==0.9.4 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: jmespath 0.9.4 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout:Adding jmespath 0.9.4 to easy-install.pth file 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for futures==3.3.0 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: futures 3.3.0 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout:Adding futures 3.3.0 to easy-install.pth file 2019-12-09T10:32:53.759 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.760 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.760 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for urllib3==1.25.7 2019-12-09T10:32:53.760 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: urllib3 1.25.7 2019-12-09T10:32:53.760 INFO:teuthology.orchestra.run.smithi033.stdout:Adding urllib3 1.25.7 to easy-install.pth file 2019-12-09T10:32:53.760 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.760 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.760 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for python-dateutil==2.8.0 2019-12-09T10:32:53.761 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: python-dateutil 2.8.0 2019-12-09T10:32:53.761 INFO:teuthology.orchestra.run.smithi033.stdout:Adding python-dateutil 2.8.0 to easy-install.pth file 2019-12-09T10:32:53.761 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.761 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.761 INFO:teuthology.orchestra.run.smithi033.stdout:Searching for docutils==0.15.2 2019-12-09T10:32:53.761 INFO:teuthology.orchestra.run.smithi033.stdout:Best match: docutils 0.15.2 2019-12-09T10:32:53.761 INFO:teuthology.orchestra.run.smithi033.stdout:Adding docutils 0.15.2 to easy-install.pth file 2019-12-09T10:32:53.762 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T10:32:53.762 INFO:teuthology.orchestra.run.smithi033.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T10:32:53.762 INFO:teuthology.orchestra.run.smithi033.stdout:Finished processing dependencies for s3tests==0.0.1 2019-12-09T10:32:53.768 INFO:teuthology.orchestra.run.smithi033:> cat > /home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2019-12-09T10:32:53.819 INFO:tasks.s3tests:Configuring boto... 2019-12-09T10:32:53.819 INFO:teuthology.orchestra.run.smithi033:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T10:32:53.915 INFO:teuthology.orchestra.run.smithi033:s3 tests against rgw> S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain' 2019-12-09T10:32:59.241 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok 2019-12-09T10:33:02.439 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok 2019-12-09T10:33:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok 2019-12-09T10:33:08.439 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok 2019-12-09T10:33:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_none ... ok 2019-12-09T10:33:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_contentlength_none ... ok 2019-12-09T10:33:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_acl_create_contentlength_none ... ok 2019-12-09T10:33:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_contentlength_none ... ok 2019-12-09T10:33:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_empty ... ok 2019-12-09T10:33:21.106 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_none ... ok 2019-12-09T10:33:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 ... ok 2019-12-09T10:33:25.842 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 ... ok 2019-12-09T10:33:29.065 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T10:33:31.531 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws2 ... ok 2019-12-09T10:33:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T10:33:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws2 ... ok 2019-12-09T10:33:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws4 ... SKIP 2019-12-09T10:33:31.858 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws4 ... SKIP 2019-12-09T10:33:31.859 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws4 ... SKIP 2019-12-09T10:33:31.859 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T10:33:31.859 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T10:33:31.859 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_none_aws4 ... SKIP 2019-12-09T10:33:31.859 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T10:33:31.859 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T10:33:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_empty_aws4 ... SKIP 2019-12-09T10:33:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T10:33:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws4 ... SKIP 2019-12-09T10:33:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T10:33:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T10:33:31.860 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T10:33:31.861 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T10:33:31.861 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T10:33:31.861 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T10:33:31.861 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T10:33:31.861 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_end_aws4 ... SKIP 2019-12-09T10:33:31.861 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_end_aws4 ... SKIP 2019-12-09T10:33:31.862 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_custom_header_aws4 ... SKIP 2019-12-09T10:33:31.862 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_header_aws4 ... SKIP 2019-12-09T10:33:31.862 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T10:33:31.862 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T10:33:31.862 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_none_aws4 ... SKIP 2019-12-09T10:33:31.862 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T10:33:31.862 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T10:33:31.863 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_empty_aws4 ... SKIP 2019-12-09T10:33:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T10:33:31.864 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws4 ... SKIP 2019-12-09T10:33:31.866 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T10:33:31.866 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T10:33:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T10:33:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T10:33:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T10:33:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T10:33:31.867 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T10:33:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_versioning_obj_read_not_exist_null ... ok 2019-12-09T10:33:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_lifecycle_transition ... SKIP 2019-12-09T10:33:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_lifecycle_transition_single_rule_multi_trans ... SKIP 2019-12-09T10:33:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_lifecycle_set_noncurrent_transition ... SKIP 2019-12-09T10:33:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_object_storage_class ... SKIP 2019-12-09T10:33:38.408 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_object_storage_class_multipart ... SKIP 2019-12-09T10:33:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_object_modify_storage_class ... SKIP 2019-12-09T10:33:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_object_modify_storage_class_multipart ... SKIP 2019-12-09T10:33:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_object_storage_class_copy ... SKIP 2019-12-09T10:33:38.409 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_object_storage_class_copy_multipart ... SKIP 2019-12-09T10:33:45.213 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ok 2019-12-09T10:33:51.444 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ok 2019-12-09T10:33:59.465 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok 2019-12-09T10:34:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok 2019-12-09T10:34:08.944 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists ... ok 2019-12-09T10:34:12.804 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_enc ... ok 2019-12-09T10:34:16.927 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag ... ok 2019-12-09T10:34:24.580 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.check_can_test_website ... SKIP 2019-12-09T10:34:24.580 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_nonexistant_bucket_rgw ... SKIP 2019-12-09T10:34:24.580 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_public_index ... SKIP 2019-12-09T10:34:24.581 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_public_index ... SKIP 2019-12-09T10:34:24.582 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty ... SKIP 2019-12-09T10:34:24.583 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty ... SKIP 2019-12-09T10:34:24.583 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index ... SKIP 2019-12-09T10:34:24.583 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index ... SKIP 2019-12-09T10:34:24.583 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T10:34:24.584 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T10:34:24.584 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T10:34:24.584 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T10:34:24.584 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T10:34:24.587 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T10:34:24.587 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T10:34:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T10:34:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T10:34:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T10:34:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T10:34:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T10:34:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_base ... SKIP 2019-12-09T10:34:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path ... SKIP 2019-12-09T10:34:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path_upgrade ... SKIP 2019-12-09T10:34:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_xredirect_nonwebsite ... SKIP 2019-12-09T10:34:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_relative ... SKIP 2019-12-09T10:34:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_abs ... SKIP 2019-12-09T10:34:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_relative ... SKIP 2019-12-09T10:34:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_abs ... SKIP 2019-12-09T10:34:24.589 INFO:teuthology.orchestra.run.smithi033.stderr:SKIP 2019-12-09T10:34:24.607 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.functional.test_utils.test_generate ... ok 2019-12-09T10:36:16.350 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T10:36:16.350 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T10:36:16.350 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T10:36:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T10:36:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T10:36:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T10:36:16.351 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T10:36:16.724 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:07:38.182 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.749 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:07:38.207 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.755 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:07:38.214 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.756 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T10:36:16.771 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:07:38.230 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.805 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:07:38.288 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.810 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:07:38.293 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.901 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:07:38.359 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.919 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:07:38.378 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.933 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:07:38.415 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:36:16.960 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T10:36:17.384 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T10:36:17.409 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T10:36:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T10:36:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T10:36:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T10:36:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T10:36:17.429 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T10:36:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T10:36:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T10:36:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T10:36:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T10:36:17.430 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T10:36:17.431 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T10:36:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T10:36:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T10:36:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T10:36:17.435 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T10:36:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T10:36:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T10:36:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T10:36:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T10:36:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T10:36:17.488 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T10:36:17.488 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T10:36:17.651 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T10:36:21.867 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short ... ok 2019-12-09T10:36:35.819 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad ... ok 2019-12-09T10:36:37.737 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty ... ok 2019-12-09T10:36:41.722 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none ... ok 2019-12-09T10:36:47.166 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch ... ok 2019-12-09T10:36:51.002 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty ... ok 2019-12-09T10:36:56.849 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none ... ok 2019-12-09T10:36:58.979 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative ... ok 2019-12-09T10:37:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid ... ok 2019-12-09T10:37:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty ... ok 2019-12-09T10:37:11.650 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none ... ok 2019-12-09T10:37:14.693 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl ... ok 2019-12-09T10:37:14.828 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:08:36.287 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:14.837 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:08:36.296 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:14.844 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:08:36.302 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:14.849 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:08:36.308 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:14.850 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:08:36.332 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:14.856 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:08:36.339 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:14.909 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:08:36.368 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:14.911 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:08:36.369 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:15.034 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:08:36.517 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:37:16.795 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch ... ok 2019-12-09T10:37:18.967 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty ... ok 2019-12-09T10:37:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative ... ok 2019-12-09T10:37:21.134 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 ... ok 2019-12-09T10:37:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 ... ok 2019-12-09T10:37:32.945 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 ... ok 2019-12-09T10:37:35.079 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 ... ok 2019-12-09T10:37:37.582 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 ... ok 2019-12-09T10:37:39.044 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 ... ok 2019-12-09T10:37:41.936 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 ... ok 2019-12-09T10:37:44.523 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 ... ok 2019-12-09T10:37:47.807 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 ... ok 2019-12-09T10:37:50.248 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 ... ok 2019-12-09T10:37:50.352 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 ... ok 2019-12-09T10:37:50.394 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 ... ok 2019-12-09T10:37:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 ... ok 2019-12-09T10:37:50.570 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 ... ok 2019-12-09T10:37:50.606 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 ... ok 2019-12-09T10:37:53.305 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_empty ... ok 2019-12-09T10:38:02.040 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_distinct ... ok 2019-12-09T10:38:09.167 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_many ... ok 2019-12-09T10:38:15.767 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_many ... ok 2019-12-09T10:38:24.826 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_basic_key_count ... ok 2019-12-09T10:38:33.485 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_basic ... ok 2019-12-09T10:38:40.691 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_basic ... ok 2019-12-09T10:38:52.101 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix ... ok 2019-12-09T10:39:05.224 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix ... ok 2019-12-09T10:39:10.040 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_ends_with_delimiter ... ok 2019-12-09T10:39:13.438 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_ends_with_delimiter ... ok 2019-12-09T10:39:24.129 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_alt ... ok 2019-12-09T10:39:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_alt ... ok 2019-12-09T10:39:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_underscore ... ok 2019-12-09T10:39:57.469 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_underscore ... ok 2019-12-09T10:40:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_percentage ... ok 2019-12-09T10:40:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_percentage ... ok 2019-12-09T10:40:18.608 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_whitespace ... ok 2019-12-09T10:40:26.133 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_whitespace ... ok 2019-12-09T10:40:33.699 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_dot ... ok 2019-12-09T10:40:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_dot ... ok 2019-12-09T10:40:47.351 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_unreadable ... ok 2019-12-09T10:40:52.303 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_unreadable ... ok 2019-12-09T10:40:59.058 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_empty ... ok 2019-12-09T10:41:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_empty ... ok 2019-12-09T10:41:13.562 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_none ... ok 2019-12-09T10:41:19.042 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_none ... ok 2019-12-09T10:41:23.795 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_notempty ... ok 2019-12-09T10:41:29.184 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_defaultempty ... ok 2019-12-09T10:41:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_empty ... ok 2019-12-09T10:41:42.018 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_exist ... ok 2019-12-09T10:41:48.880 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_not_exist ... ok 2019-12-09T10:42:26.876 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:13:48.335 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.884 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:13:48.344 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.892 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:13:48.375 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.893 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:13:48.353 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.896 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:13:48.379 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:26.897 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:13:48.356 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:27.084 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:13:48.567 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:27.109 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:13:48.569 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:42:27.144 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:13:48.602 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.267 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:15:59.726 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.304 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:15:59.763 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.344 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:15:59.826 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.347 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:15:59.830 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.389 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:15:59.848 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.389 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:15:59.848 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.395 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:15:59.854 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.433 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:15:59.916 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:44:38.472 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:15:59.931 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:38.904 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:19:00.363 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:38.913 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:19:00.373 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:38.923 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:19:00.382 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:38.926 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:19:00.409 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:38.932 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:19:00.415 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:38.932 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:19:00.391 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:39.035 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:19:00.495 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:39.062 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:19:00.545 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:47:39.067 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:19:00.526 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.216 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:24:12.675 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.221 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:24:12.681 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.231 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:24:12.690 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.232 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:24:12.692 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.236 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:24:12.719 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.244 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:24:12.727 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.424 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:24:12.883 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.426 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:24:12.885 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:51.457 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:24:12.940 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:59.918 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:24:21.377 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:59.923 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:24:21.383 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:59.941 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:24:21.400 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:59.949 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:24:21.408 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:59.955 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:24:21.414 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:52:59.998 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:24:21.457 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:00.011 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:24:21.494 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:00.019 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:24:21.503 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:53:00.053 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:24:21.536 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.431 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:29:24.890 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.439 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:29:24.898 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.449 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:29:24.908 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.457 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:29:24.916 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.463 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:29:24.946 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.486 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:29:24.968 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.671 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:29:25.130 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.701 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:29:25.155 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:03.741 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:29:25.224 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T10:58:49.518 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_skip_special ... ok 2019-12-09T10:58:55.751 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_basic ... ok 2019-12-09T10:59:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_basic ... ok 2019-12-09T10:59:09.787 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_alt ... ok 2019-12-09T10:59:14.199 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_alt ... ok 2019-12-09T10:59:20.918 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_empty ... ok 2019-12-09T10:59:28.001 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_empty ... ok 2019-12-09T10:59:34.568 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_none ... ok 2019-12-09T10:59:43.037 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_none ... ok 2019-12-09T10:59:49.735 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_not_exist ... ok 2019-12-09T10:59:57.289 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_not_exist ... ok 2019-12-09T11:00:02.702 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_unreadable ... ok 2019-12-09T11:00:10.922 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_unreadable ... ok 2019-12-09T11:00:17.304 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_basic ... ok 2019-12-09T11:00:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_basic ... ok 2019-12-09T11:00:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_alt ... ok 2019-12-09T11:00:38.783 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_alt ... ok 2019-12-09T11:00:45.728 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_not_exist ... ok 2019-12-09T11:00:51.986 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_not_exist ... ok 2019-12-09T11:00:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_delimiter_not_exist ... ok 2019-12-09T11:01:05.695 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_delimiter_not_exist ... ok 2019-12-09T11:01:11.047 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-12-09T11:01:18.914 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-12-09T11:01:21.405 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:32:42.864 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.443 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:32:42.902 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.448 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:32:42.907 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.453 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:32:42.936 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.456 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:32:42.939 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.468 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:32:42.927 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.547 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:32:43.030 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.559 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:32:43.018 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:21.596 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:32:43.055 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:01:25.216 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_one ... ok 2019-12-09T11:01:31.582 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_one ... ok 2019-12-09T11:01:37.736 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_zero ... ok 2019-12-09T11:01:43.968 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_zero ... ok 2019-12-09T11:01:50.755 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_none ... ok 2019-12-09T11:01:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_none ... ok 2019-12-09T11:02:31.498 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_unordered ... ok 2019-12-09T11:03:02.142 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_unordered ... ok 2019-12-09T11:03:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_invalid ... ok 2019-12-09T11:03:13.503 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:34:34.962 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.508 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:34:34.967 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.517 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:34:34.976 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.518 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:34:35.001 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.522 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:34:35.005 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.524 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:34:34.983 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.596 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:34:35.055 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.618 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:34:35.078 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:13.620 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:34:35.103 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:03:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_none ... ok 2019-12-09T11:03:25.483 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_empty ... ok 2019-12-09T11:03:32.934 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_continuationtoken_empty ... ok 2019-12-09T11:03:42.844 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_unreadable ... ok 2019-12-09T11:03:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_unreadable ... ok 2019-12-09T11:03:58.433 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_not_in_list ... ok 2019-12-09T11:04:03.857 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_not_in_list ... ok 2019-12-09T11:04:12.660 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_after_list ... ok 2019-12-09T11:04:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_after_list ... ok 2019-12-09T11:04:31.166 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data ... ok 2019-12-09T11:04:50.340 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning ... ok 2019-12-09T11:04:55.335 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous ... ok 2019-12-09T11:04:59.636 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous ... ok 2019-12-09T11:05:02.083 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous_fail ... ok 2019-12-09T11:05:04.725 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous_fail ... ok 2019-12-09T11:05:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_notexist ... ok 2019-12-09T11:05:05.139 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_notexist ... ok 2019-12-09T11:05:05.435 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_notexist ... ok 2019-12-09T11:05:10.362 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty ... ok 2019-12-09T11:05:16.658 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl ... ok 2019-12-09T11:05:17.398 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket ... ok 2019-12-09T11:05:23.104 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_delete ... ok 2019-12-09T11:05:25.718 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_read_notexist ... ok 2019-12-09T11:05:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_requestid_matches_header_on_error ... ok 2019-12-09T11:05:40.279 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multi_object_delete ... ok 2019-12-09T11:05:52.114 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multi_objectv2_delete ... ok 2019-12-09T11:05:56.167 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_head_zero_bytes ... ok 2019-12-09T11:06:01.597 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_write_check_etag ... ok 2019-12-09T11:06:07.208 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_write_cache_control ... ok 2019-12-09T11:06:10.701 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_write_expires ... ok 2019-12-09T11:06:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete ... ok 2019-12-09T11:06:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good ... ok 2019-12-09T11:06:27.352 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty ... ok 2019-12-09T11:06:33.681 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty ... ok 2019-12-09T11:06:39.348 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_unicode_metadata ... ok 2019-12-09T11:06:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put ... ok 2019-12-09T11:06:49.346 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_write_file ... ok 2019-12-09T11:06:53.520 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_anonymous_request ... ok 2019-12-09T11:06:57.000 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request ... ok 2019-12-09T11:07:02.120 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_no_content_type ... ok 2019-12-09T11:07:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request_bad_access_key ... ok 2019-12-09T11:07:11.472 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_success_code ... ok 2019-12-09T11:07:13.875 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_invalid_success_code ... ok 2019-12-09T11:07:20.102 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_larger_than_chunk ... ok 2019-12-09T11:07:24.725 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_key_from_filename ... ok 2019-12-09T11:07:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_ignored_header ... ok 2019-12-09T11:07:32.839 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_case_insensitive_condition_fields ... ok 2019-12-09T11:07:37.478 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_escaped_field_values ... ok 2019-12-09T11:07:42.716 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_success_redirect_action ... ok 2019-12-09T11:07:45.092 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_signature ... ok 2019-12-09T11:07:47.851 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_access_key ... ok 2019-12-09T11:07:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_date_format ... ok 2019-12-09T11:07:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_no_key_specified ... ok 2019-12-09T11:07:57.211 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_signature ... ok 2019-12-09T11:07:59.563 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_policy_condition ... ok 2019-12-09T11:08:03.657 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_user_specified_header ... ok 2019-12-09T11:08:05.796 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_request_missing_policy_specified_field ... ok 2019-12-09T11:08:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_condition_is_case_sensitive ... ok 2019-12-09T11:08:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_expires_is_case_sensitive ... ok 2019-12-09T11:08:14.595 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_expired_policy ... ok 2019-12-09T11:08:16.922 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_request_field_value ... ok 2019-12-09T11:08:18.950 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_expires_condition ... ok 2019-12-09T11:08:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_conditions_list ... ok 2019-12-09T11:08:24.971 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_limit_exceeded ... ok 2019-12-09T11:08:25.859 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:39:47.318 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:25.865 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:39:47.324 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:25.879 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:39:47.362 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:25.880 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:39:47.339 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:25.884 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:39:47.343 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:25.885 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:39:47.368 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:26.076 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:39:47.535 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:26.078 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:39:47.537 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:26.080 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:39:47.563 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:08:27.248 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_content_length_argument ... ok 2019-12-09T11:08:29.981 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_content_length_argument ... ok 2019-12-09T11:08:32.220 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_below_minimum ... ok 2019-12-09T11:08:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_empty_conditions ... ok 2019-12-09T11:08:38.516 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_good ... ok 2019-12-09T11:08:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_failed ... ok 2019-12-09T11:08:47.959 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_good ... ok 2019-12-09T11:08:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_failed ... ok 2019-12-09T11:08:56.743 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_good ... ok 2019-12-09T11:09:03.559 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_failed ... ok 2019-12-09T11:09:08.941 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_good ... ok 2019-12-09T11:09:13.749 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_failed ... ok 2019-12-09T11:09:20.062 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good ... ok 2019-12-09T11:09:26.225 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed ... ok 2019-12-09T11:09:32.153 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... ok 2019-12-09T11:09:36.004 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... ok 2019-12-09T11:09:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good ... ok 2019-12-09T11:09:42.946 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:41:04.406 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:42.951 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:41:04.410 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:42.991 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:41:04.450 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:42.993 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:41:04.452 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:43.004 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:41:04.464 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:43.035 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:41:04.519 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:43.042 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:41:04.501 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:43.043 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:41:04.527 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:43.146 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:41:04.629 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:09:49.555 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed ... ok 2019-12-09T11:09:53.084 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... ok 2019-12-09T11:09:57.426 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... ok 2019-12-09T11:10:01.813 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get ... ok 2019-12-09T11:10:10.219 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_gone ... ok 2019-12-09T11:10:22.374 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_delete_key_bucket_gone ... ok 2019-12-09T11:10:26.720 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_gone ... ok 2019-12-09T11:10:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_head ... ok 2019-12-09T11:10:38.316 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:41:59.775 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.334 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:41:59.793 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.371 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:41:59.854 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.388 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:41:59.871 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.388 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:41:59.847 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.389 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:41:59.848 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.420 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:41:59.879 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.449 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:41:59.908 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:38.453 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:41:59.937 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:10:39.218 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_head_extended ... ok 2019-12-09T11:10:42.363 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_acl ... ok 2019-12-09T11:10:46.153 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_acl ... ok 2019-12-09T11:10:51.342 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated ... ok 2019-12-09T11:10:55.441 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_response_headers ... ok 2019-12-09T11:10:57.677 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_acl ... ok 2019-12-09T11:11:00.991 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_acl ... ok 2019-12-09T11:11:09.544 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_gone ... ok 2019-12-09T11:11:13.480 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_gone ... ok 2019-12-09T11:11:16.775 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_not_expired ... ok 2019-12-09T11:11:20.487 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_range_zero ... ok 2019-12-09T11:11:23.509 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_max_range ... ok 2019-12-09T11:11:26.601 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_positive_range ... ok 2019-12-09T11:11:28.375 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put ... ok 2019-12-09T11:11:32.236 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put_write_access ... ok 2019-12-09T11:11:35.371 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_put_authenticated ... ok 2019-12-09T11:11:38.606 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired ... ok 2019-12-09T11:11:39.595 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one ... ok 2019-12-09T11:11:40.102 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two ... ok 2019-12-09T11:11:40.531 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip ... ok 2019-12-09T11:11:41.280 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore ... ok 2019-12-09T11:11:45.881 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists ... ok 2019-12-09T11:11:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_get_location ... ok 2019-12-09T11:11:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner ... ok 2019-12-09T11:11:53.734 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_default ... ok 2019-12-09T11:11:56.714 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_during_create ... ok 2019-12-09T11:12:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned ... ok 2019-12-09T11:12:03.176 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_publicreadwrite ... ok 2019-12-09T11:12:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_authenticatedread ... ok 2019-12-09T11:12:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_default ... ok 2019-12-09T11:12:12.598 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_during_create ... ok 2019-12-09T11:12:17.922 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned ... ok 2019-12-09T11:12:21.933 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_publicreadwrite ... ok 2019-12-09T11:12:26.038 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_authenticatedread ... ok 2019-12-09T11:12:32.985 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerread ... ok 2019-12-09T11:12:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerfullcontrol ... ok 2019-12-09T11:12:45.735 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_owner ... ok 2019-12-09T11:12:50.298 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_attributes ... ok 2019-12-09T11:12:53.697 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_private_to_private ... ok 2019-12-09T11:13:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl ... ok 2019-12-09T11:13:06.190 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_write ... ok 2019-12-09T11:13:11.836 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_writeacp ... ok 2019-12-09T11:13:18.197 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_read ... ok 2019-12-09T11:13:23.255 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_acl_readacp ... ok 2019-12-09T11:13:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_fullcontrol ... ok 2019-12-09T11:13:35.419 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_read ... ok 2019-12-09T11:13:37.877 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:44:59.336 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:37.881 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:44:59.341 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:37.893 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:44:59.376 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:37.894 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:44:59.352 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:37.897 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:44:59.356 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:37.899 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:44:59.382 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:38.016 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:44:59.475 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:38.036 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:44:59.495 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:38.040 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:44:59.524 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:13:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_readacp ... ok 2019-12-09T11:13:45.066 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_write ... ok 2019-12-09T11:13:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_writeacp ... ok 2019-12-09T11:13:53.219 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_nonexist_user ... ok 2019-12-09T11:14:01.870 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_no_grants ... ok 2019-12-09T11:14:06.265 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_header_acl_grants ... ok 2019-12-09T11:14:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_header_acl_grants ... ok 2019-12-09T11:14:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email ... ok 2019-12-09T11:14:18.836 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email_notexist ... ok 2019-12-09T11:14:24.597 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_revoke_all ... ok 2019-12-09T11:14:33.883 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_private ... ok 2019-12-09T11:14:41.824 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_private ... ok 2019-12-09T11:14:51.158 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicread ... ok 2019-12-09T11:15:00.233 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicread ... ok 2019-12-09T11:15:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicreadwrite ... ok 2019-12-09T11:15:20.501 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicreadwrite ... ok 2019-12-09T11:15:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_private ... ok 2019-12-09T11:15:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicread ... ok 2019-12-09T11:15:49.665 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicreadwrite ... ok 2019-12-09T11:16:01.432 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_private ... ok 2019-12-09T11:16:15.848 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicread ... ok 2019-12-09T11:16:28.066 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicreadwrite ... ok 2019-12-09T11:16:43.791 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_buckets_create_then_list ... ok 2019-12-09T11:16:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_anonymous ... ok 2019-12-09T11:16:45.922 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth ... ok 2019-12-09T11:16:46.036 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth ... ok 2019-12-09T11:16:56.147 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha ... ok 2019-12-09T11:17:06.636 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit ... ok 2019-12-09T11:17:09.452 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period ... ok 2019-12-09T11:17:12.285 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... ok 2019-12-09T11:17:21.172 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding ... ok 2019-12-09T11:17:55.978 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names ... ok 2019-12-09T11:18:05.484 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_special_prefix ... ok 2019-12-09T11:18:09.634 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:49:31.093 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.678 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:49:31.137 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.680 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:49:31.139 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.693 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:49:31.152 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.704 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:49:31.187 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.705 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:49:31.189 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.795 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:49:31.255 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.822 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:49:31.281 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:09.837 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:49:31.321 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_zero_size ... ok 2019-12-09T11:18:15.840 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_same_bucket ... ok 2019-12-09T11:18:22.148 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_verify_contenttype ... ok 2019-12-09T11:18:26.995 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself ... ok 2019-12-09T11:18:32.684 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself_with_metadata ... ok 2019-12-09T11:18:42.085 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_diff_bucket ... ok 2019-12-09T11:18:46.323 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:50:07.782 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.326 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:50:07.785 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.346 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:50:07.830 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.352 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:50:07.836 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.369 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:50:07.828 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.372 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:50:07.831 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.376 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:50:07.835 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.422 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:50:07.906 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:46.437 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:50:07.896 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:18:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_bucket ... ok 2019-12-09T11:18:59.108 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_object_bucket ... ok 2019-12-09T11:19:07.482 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_canned_acl ... ok 2019-12-09T11:19:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_retaining_metadata ... ok 2019-12-09T11:19:30.598 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_replacing_metadata ... ok 2019-12-09T11:19:32.795 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_bucket_not_found ... ok 2019-12-09T11:19:35.520 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_key_not_found ... ok 2019-12-09T11:20:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket ... ok 2019-12-09T11:20:24.362 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding ... ok 2019-12-09T11:20:58.029 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:20:57.971+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:21:23.969 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload ... ok 2019-12-09T11:21:28.145 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_empty ... ok 2019-12-09T11:21:36.638 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_small ... ok 2019-12-09T11:21:55.308 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:21:55.270+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:21:55.511 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_small ... ok 2019-12-09T11:22:00.000 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range ... ok 2019-12-09T11:22:12.186 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_without_range ... ok 2019-12-09T11:22:26.307 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:22:26.295+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:22:41.916 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:22:41.784+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:23:16.504 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:23:16.491+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:23:17.055 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_special_names ... ok 2019-12-09T11:23:47.868 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:23:47.818+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:24:00.395 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:55:21.854 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.406 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:55:21.864 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.406 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:55:21.864 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.409 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:55:21.868 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.414 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:55:21.898 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.424 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:55:21.908 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.594 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:55:22.052 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.605 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:55:22.089 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:00.608 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:55:22.067 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:24:15.603 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:24:15.552+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:24:17.766 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload ... ok 2019-12-09T11:25:30.929 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:25:30.610+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:25:54.485 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:25:54.455+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:25:56.996 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:25:56.969+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:25:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_versioned ... ok 2019-12-09T11:25:58.363 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:25:58.321+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:26:31.329 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:57:52.788 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.370 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:57:52.828 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.417 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:57:52.901 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.424 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:57:52.907 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.425 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:57:52.883 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.430 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:57:52.888 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.452 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:01:57:52.910 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.508 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:01:57:52.967 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:26:31.513 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:01:57:52.997 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:28:02.590 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:28:02.550+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:28:36.456 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:28:36.390+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:29:08.539 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:00:29.997 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.548 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:00:30.007 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.554 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:00:30.013 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.560 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:00:30.018 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.560 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:00:30.044 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.571 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:00:30.055 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.688 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:00:30.171 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.694 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:00:30.153 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:29:08.701 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:00:30.159 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:07.807 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:32:07.742+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:32:33.716 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:32:33.633+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:32:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part ... ok 2019-12-09T11:33:50.911 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes ... ok 2019-12-09T11:34:09.987 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:34:09.953+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:34:20.732 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:05:42.188 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.739 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:05:42.198 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.750 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:05:42.206 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.757 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:05:42.237 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.759 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:05:42.213 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.772 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:05:42.250 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.945 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:05:42.403 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.955 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:05:42.413 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:20.964 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:05:42.447 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:23.456 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:34:23.419+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:34:38.131 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:34:38.062+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:34:52.995 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:06:14.453 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:52.997 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:06:14.455 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:53.041 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:06:14.498 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:53.041 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:06:14.499 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:53.055 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:06:14.538 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:53.059 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:06:14.541 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:53.064 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:06:14.519 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:53.101 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:06:14.582 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:53.109 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:06:14.565 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:34:56.656 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:34:56.621+0000 40262700 -1 handle_data failed with exit code -5 2019-12-09T11:35:15.601 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:35:15.541+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:35:30.338 INFO:tasks.rgw.client.0.smithi033.stdout:2019-12-09T11:35:30.299+0000 3fa61700 -1 handle_data failed with exit code -5 2019-12-09T11:35:30.932 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes ... ok 2019-12-09T11:35:56.740 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small ... ok 2019-12-09T11:36:16.904 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_contents ... ok 2019-12-09T11:36:35.165 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object ... ok 2019-12-09T11:36:48.327 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload ... ok 2019-12-09T11:36:51.553 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found ... ok 2019-12-09T11:37:17.072 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_list_multipart_upload ... ok 2019-12-09T11:37:26.722 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part ... ok 2019-12-09T11:37:34.800 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag ... ok 2019-12-09T11:37:38.815 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_100_continue ... ok 2019-12-09T11:37:45.542 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_set_cors ... ok 2019-12-09T11:38:10.941 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_response ... ok 2019-12-09T11:38:19.993 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_wildcard ... ok 2019-12-09T11:38:28.180 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_cors_header_option ... ok 2019-12-09T11:38:38.317 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_1mb ... ok 2019-12-09T11:38:48.007 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_4mb ... ok 2019-12-09T11:38:58.965 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_8mb ... ok 2019-12-09T11:39:09.085 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_1mb ... ok 2019-12-09T11:39:21.103 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_4mb ... ok 2019-12-09T11:39:28.982 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:10:50.440 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:28.990 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:10:50.448 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:28.999 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:10:50.483 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:29.000 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:10:50.457 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:29.003 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:10:50.461 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:29.005 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:10:50.488 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:29.184 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:10:50.642 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:29.191 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:10:50.674 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:29.192 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:10:50.650 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:39:36.029 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_8mb ... ok 2019-12-09T11:39:48.808 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb ... ok 2019-12-09T11:40:05.608 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb ... ok 2019-12-09T11:40:32.143 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb ... ok 2019-12-09T11:40:39.976 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb ... ok 2019-12-09T11:40:46.836 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write ... ok 2019-12-09T11:41:06.298 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last ... ok 2019-12-09T11:41:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_response_code ... ok 2019-12-09T11:41:19.735 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_ranged_big_request_response_code ... ok 2019-12-09T11:41:23.684 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_skip_leading_bytes_response_code ... ok 2019-12-09T11:41:27.278 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_return_trailing_bytes_response_code ... ok 2019-12-09T11:41:31.590 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_invalid_range ... ok 2019-12-09T11:41:35.218 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_empty_object ... ok 2019-12-09T11:41:40.693 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend ... ok 2019-12-09T11:43:14.528 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:14:35.986 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.535 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:14:35.993 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.556 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:14:36.039 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.575 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:14:36.058 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.583 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:14:36.041 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.587 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:14:36.045 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.669 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:14:36.152 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.741 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:14:36.198 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:14.828 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:14:36.286 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:39.855 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:16:01.313 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:39.861 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:16:01.319 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:39.879 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:16:01.336 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:39.880 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:16:01.363 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:39.886 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:16:01.344 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:39.888 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:16:01.372 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:39.997 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:16:01.455 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:40.003 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:16:01.461 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:44:40.008 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:16:01.491 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:38.290 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove ... ok 2019-12-09T11:47:28.571 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head ... ok 2019-12-09T11:47:40.901 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal ... ok 2019-12-09T11:48:01.751 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite ... ok 2019-12-09T11:48:16.948 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended ... ok 2019-12-09T11:49:54.820 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:21:16.278 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:54.830 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:21:16.288 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:54.835 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:21:16.293 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:54.838 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:21:16.296 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:54.849 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:21:16.333 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:54.861 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:21:16.344 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:55.069 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:21:16.527 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:55.072 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:21:16.530 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:49:55.105 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:21:16.588 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:13.114 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions ... ok 2019-12-09T11:50:56.095 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:22:17.553 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.114 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:22:17.572 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.167 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:22:17.650 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.184 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:22:17.642 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.186 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:22:17.644 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.192 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:22:17.651 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.200 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:22:17.683 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.261 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:22:17.720 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:50:56.271 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:22:17.754 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.292 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:22:57.750 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.295 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:22:57.754 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.335 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:22:57.793 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.345 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:22:57.803 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.347 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:22:57.805 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.363 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:22:57.846 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.369 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:22:57.852 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.381 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:22:57.839 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:36.444 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:22:57.928 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:51:50.231 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all ... ok 2019-12-09T11:55:04.851 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:26:26.309 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:04.860 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:26:26.318 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:04.869 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:26:26.327 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:04.875 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:26:26.334 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:04.878 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:26:26.361 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:04.890 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:26:26.373 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:05.048 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:26:26.507 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:05.054 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:26:26.537 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:55:05.068 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:26:26.526 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:04.329 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names ... ok 2019-12-09T11:59:57.790 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:31:19.249 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:57.824 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:31:19.283 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:57.825 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:31:19.284 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:57.836 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:31:19.295 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:57.861 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:31:19.343 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:57.867 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:31:19.350 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:57.960 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:31:19.419 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:57.996 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:31:19.455 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T11:59:58.040 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:31:19.523 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:14.971 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:31:36.430 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:14.984 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:31:36.443 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:14.990 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:31:36.449 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:14.994 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:31:36.477 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:14.995 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:31:36.454 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:15.005 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:31:36.488 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:15.022 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:31:36.481 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:15.036 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:31:36.495 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:15.052 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:31:36.534 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:00:30.617 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart ... ok 2019-12-09T12:01:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker ... ok 2019-12-09T12:01:38.923 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version ... ok 2019-12-09T12:01:56.635 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete ... ok 2019-12-09T12:02:26.187 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker ... ok 2019-12-09T12:02:34.261 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create ... ok 2019-12-09T12:02:55.595 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl ... ok 2019-12-09T12:03:11.396 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified ... ok 2019-12-09T12:03:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove ... Exception in thread Thread-52: 2019-12-09T12:03:17.326 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:17.326 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:17.327 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:17.327 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:17.327 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:17.327 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:17.327 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:17.327 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:17.327 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:17.328 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:17.328 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:17.328 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:17.636 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-53: 2019-12-09T12:03:17.637 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:17.637 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:17.637 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:17.637 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:17.637 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:17.637 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:17.637 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:17.638 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:17.638 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:17.638 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:17.638 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:17.638 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:17.639 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:18.359 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-54: 2019-12-09T12:03:18.359 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:18.360 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:18.360 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:18.360 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:18.360 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:18.360 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:18.360 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:18.360 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:18.361 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:18.361 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:18.361 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:18.361 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:18.361 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-55: 2019-12-09T12:03:18.367 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:18.367 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:18.367 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:18.367 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:18.367 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:18.368 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:18.368 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:18.368 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:18.368 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:18.368 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:18.368 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:18.369 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:18.369 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-62: 2019-12-09T12:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:28.558 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:28.559 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:28.560 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:28.560 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:28.560 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:28.755 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-61: 2019-12-09T12:03:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:28.756 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:28.756 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:28.756 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:28.756 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:28.756 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:28.756 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:28.757 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:28.757 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:28.757 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:28.757 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:28.757 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:28.793 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-64: 2019-12-09T12:03:28.794 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:28.794 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:28.794 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:28.794 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:28.794 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:28.794 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:28.794 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:28.795 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:28.795 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:28.795 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:28.795 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:28.795 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:28.795 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:29.539 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-63: 2019-12-09T12:03:29.539 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:29.539 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:29.539 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:29.540 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:29.540 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:29.540 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:29.540 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:29.540 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:29.540 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:29.541 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:29.541 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:29.541 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:29.541 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-72: 2019-12-09T12:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:38.033 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:38.033 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:38.042 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-75: 2019-12-09T12:03:38.043 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:38.043 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:38.043 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:38.043 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:38.043 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:38.043 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:38.043 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:38.044 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:38.044 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:38.044 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:38.044 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:38.044 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:38.044 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:38.614 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-74: 2019-12-09T12:03:38.614 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:38.614 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:38.614 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:38.615 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:38.615 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:38.615 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:38.615 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:38.615 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:38.615 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:38.615 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:38.616 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:38.616 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:38.616 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-73: 2019-12-09T12:03:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:39.187 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:39.187 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:39.187 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:39.188 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:39.188 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:39.188 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:39.188 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:39.188 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:39.188 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:39.188 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:39.189 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-85: 2019-12-09T12:03:50.925 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:50.925 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:50.926 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:50.926 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:50.926 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:50.926 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:50.926 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:50.926 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:50.927 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:50.927 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:50.927 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:50.927 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:50.927 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:50.976 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-83: 2019-12-09T12:03:50.976 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:50.976 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:50.977 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:50.977 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:50.977 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:50.977 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:50.977 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:50.977 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:50.977 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:50.978 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:50.978 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:50.978 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:51.413 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-84: 2019-12-09T12:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:03:51.423 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-82: 2019-12-09T12:03:51.424 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:03:51.424 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:03:51.424 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:03:51.424 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:03:51.424 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:03:51.424 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:03:51.424 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:03:51.425 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:03:51.425 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:03:51.425 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:03:51.425 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:03:51.425 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:03:51.425 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-93: 2019-12-09T12:04:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:02.269 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:02.270 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:02.270 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:02.270 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:02.270 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:02.271 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:02.271 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:02.271 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:02.271 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:02.271 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:02.272 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:02.487 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-92: 2019-12-09T12:04:02.488 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:02.488 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:02.488 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:02.488 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:02.488 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:02.488 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:02.489 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:02.489 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:02.489 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:02.489 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:02.489 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:02.489 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:02.489 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:02.756 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-94: 2019-12-09T12:04:02.757 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:02.757 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:02.757 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:02.757 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:02.757 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:02.757 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:02.758 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:02.758 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:02.758 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:02.758 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:02.758 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:02.758 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:02.758 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:03.224 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-95: 2019-12-09T12:04:03.225 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:03.225 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:03.225 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:03.225 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:03.226 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:03.226 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:03.226 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:03.226 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:03.227 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:03.227 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:03.227 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:03.227 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:03.228 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:12.820 INFO:teuthology.orchestra.run.smithi033.stderr:ok 2019-12-09T12:04:19.488 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove ... Exception in thread Thread-101: 2019-12-09T12:04:19.489 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:19.489 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:19.489 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:19.489 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:19.490 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:19.490 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:19.490 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:19.490 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:19.490 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:19.491 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:19.491 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:19.491 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:19.491 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-104: 2019-12-09T12:04:19.855 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:19.855 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:19.855 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:19.855 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:19.855 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:19.855 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:19.856 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:19.857 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:19.857 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-103: 2019-12-09T12:04:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:19.937 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:19.937 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:19.937 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:19.937 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:19.938 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:19.938 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:19.938 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:19.938 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:19.938 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:19.938 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:19.939 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:20.359 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-106: 2019-12-09T12:04:20.359 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:20.360 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:20.360 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:20.360 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:20.360 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:20.360 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:20.361 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:20.361 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:20.361 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:20.361 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:20.361 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:20.361 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:20.362 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:20.366 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-108: 2019-12-09T12:04:20.366 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:20.367 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:20.367 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:20.367 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:20.367 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:20.367 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:20.368 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:20.368 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:20.368 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:20.368 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:20.368 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:20.369 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:20.369 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:20.412 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-105: 2019-12-09T12:04:20.413 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:20.413 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:20.413 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:20.413 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:20.413 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:20.413 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:20.414 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:20.414 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:20.414 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:20.414 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:20.414 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:20.414 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:20.414 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:20.416 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-107: 2019-12-09T12:04:20.416 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:20.416 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:20.417 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:20.417 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:20.417 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:20.417 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:20.417 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:20.418 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:20.418 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:20.418 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:20.418 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:20.418 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:20.418 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:Exception in thread Thread-109: 2019-12-09T12:04:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:Traceback (most recent call last): 2019-12-09T12:04:20.433 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T12:04:20.433 INFO:teuthology.orchestra.run.smithi033.stderr: self.run() 2019-12-09T12:04:20.433 INFO:teuthology.orchestra.run.smithi033.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T12:04:20.433 INFO:teuthology.orchestra.run.smithi033.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T12:04:20.434 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T12:04:20.434 INFO:teuthology.orchestra.run.smithi033.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T12:04:20.434 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T12:04:20.434 INFO:teuthology.orchestra.run.smithi033.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T12:04:20.434 INFO:teuthology.orchestra.run.smithi033.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T12:04:20.434 INFO:teuthology.orchestra.run.smithi033.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T12:04:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T12:04:20.435 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T12:04:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:ok 2019-12-09T12:04:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set ... ok 2019-12-09T12:04:41.831 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get ... ok 2019-12-09T12:04:48.258 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id ... ok 2019-12-09T12:04:50.177 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long ... ok 2019-12-09T12:04:52.793 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_same_id ... ok 2019-12-09T12:04:55.717 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status ... ok 2019-12-09T12:05:01.697 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_date ... ok 2019-12-09T12:05:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date ... ok 2019-12-09T12:05:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:Check for valid x-amz-expiration header after PUT ... ok 2019-12-09T12:05:15.642 INFO:teuthology.orchestra.run.smithi033.stderr:Check for valid x-amz-expiration header on HEAD request ... ok 2019-12-09T12:05:21.983 INFO:teuthology.orchestra.run.smithi033.stderr:Check for valid x-amz-expiration header on GET request ... ok 2019-12-09T12:05:28.953 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:36:50.410 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:28.967 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:36:50.417 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:28.967 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:36:50.426 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:28.971 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:36:50.454 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:28.974 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:36:50.433 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:28.978 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:36:50.460 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:29.166 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:36:50.648 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:29.233 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:36:50.692 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:29.238 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:36:50.697 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:05:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:Check x-amz-expiration-header object tags interaction ... ok 2019-12-09T12:05:38.104 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent ... ok 2019-12-09T12:05:41.775 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker ... ok 2019-12-09T12:05:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_filter ... ok 2019-12-09T12:05:52.398 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter ... ok 2019-12-09T12:05:56.271 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart ... ok 2019-12-09T12:06:02.064 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1b ... ok 2019-12-09T12:06:06.949 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1kb ... ok 2019-12-09T12:06:11.358 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1MB ... ok 2019-12-09T12:06:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_13b ... ok 2019-12-09T12:06:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... ok 2019-12-09T12:06:25.166 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... ok 2019-12-09T12:06:29.290 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... ok 2019-12-09T12:06:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... ok 2019-12-09T12:06:33.873 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... ok 2019-12-09T12:06:36.108 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... ok 2019-12-09T12:06:39.941 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... ok 2019-12-09T12:08:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... ok 2019-12-09T12:08:24.335 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:39:45.795 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.382 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:39:45.841 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.383 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:39:45.843 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.398 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:39:45.857 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.440 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:39:45.921 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.445 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:39:45.928 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.471 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:39:45.931 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.482 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:39:45.942 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:24.542 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:39:46.025 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:48.279 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... ok 2019-12-09T12:08:52.541 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... ok 2019-12-09T12:08:58.267 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_method_head ... ok 2019-12-09T12:09:01.672 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_present ... ok 2019-12-09T12:09:04.585 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_no_key ... ok 2019-12-09T12:09:06.778 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_not_declared ... ok 2019-12-09T12:10:27.777 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_upload ... ok 2019-12-09T12:10:41.749 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:42:03.209 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.757 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:42:03.217 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.773 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:42:03.254 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.773 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:42:03.233 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.777 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:42:03.260 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.779 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:42:03.239 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.860 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:42:03.320 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.899 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:42:03.354 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:41.940 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:42:03.422 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:10:58.929 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_1 ... ok 2019-12-09T12:11:28.713 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_2 ... ok 2019-12-09T12:11:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_post_object_authenticated_request ... ok 2019-12-09T12:11:37.578 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1b ... ok 2019-12-09T12:11:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1kb ... ok 2019-12-09T12:11:45.566 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1MB ... ok 2019-12-09T12:11:48.914 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_13b ... ok 2019-12-09T12:11:53.706 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_read_declare ... ok 2019-12-09T12:11:59.948 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy ... ok 2019-12-09T12:12:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy ... ok 2019-12-09T12:12:09.410 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_acl ... ok 2019-12-09T12:12:16.563 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl ... ok 2019-12-09T12:12:25.530 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket ... ok 2019-12-09T12:12:36.629 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket ... ok 2019-12-09T12:12:44.413 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_obj_tagging ... ok 2019-12-09T12:12:51.857 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_obj_head_tagging ... ok 2019-12-09T12:12:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_max_tags ... ok 2019-12-09T12:13:04.984 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_excess_tags ... ok 2019-12-09T12:13:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags ... ok 2019-12-09T12:13:18.049 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_excess_key_tags ... ok 2019-12-09T12:13:24.117 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_excess_val_tags ... ok 2019-12-09T12:13:33.199 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_modify_tags ... ok 2019-12-09T12:13:42.893 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_delete_tags ... ok 2019-12-09T12:13:47.480 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_anonymous_request ... ok 2019-12-09T12:13:52.256 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_authenticated_request ... ok 2019-12-09T12:13:56.545 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_obj_with_tags ... ok 2019-12-09T12:14:03.722 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_get_tags_acl_public ... ok 2019-12-09T12:14:11.140 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_put_tags_acl_public ... ok 2019-12-09T12:14:19.732 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_delete_tags_obj_public ... ok 2019-12-09T12:14:35.388 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id ... ok 2019-12-09T12:15:53.941 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:47:15.401 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:53.949 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:47:15.409 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:53.957 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:47:15.417 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:53.958 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:47:15.441 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:53.962 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:47:15.422 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:53.962 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:47:15.445 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:54.148 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:47:15.631 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:54.153 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:47:15.613 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:15:54.153 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:47:15.613 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:00.741 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id ... ok 2019-12-09T12:16:11.731 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag ... ok 2019-12-09T12:16:22.839 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag ... ok 2019-12-09T12:16:34.284 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag ... ok 2019-12-09T12:16:45.938 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:48:07.398 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:45.976 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:48:07.436 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:45.978 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:48:07.439 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:45.987 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:48:07.447 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:46.022 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:48:07.505 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:46.025 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:48:07.486 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:46.026 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:48:07.487 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:46.035 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:48:07.518 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:46.097 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:48:07.580 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:16:47.483 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source ... ok 2019-12-09T12:17:00.956 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta ... ok 2019-12-09T12:17:06.031 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl ... ok 2019-12-09T12:17:18.838 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant ... ok 2019-12-09T12:17:28.865 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag ... ok 2019-12-09T12:17:34.820 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock ... ok 2019-12-09T12:17:37.227 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_bucket ... ok 2019-12-09T12:17:39.774 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_with_days_and_years ... ok 2019-12-09T12:17:43.965 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_days ... ok 2019-12-09T12:17:47.398 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_years ... ok 2019-12-09T12:17:49.627 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_status ... ok 2019-12-09T12:17:52.050 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_suspend_versioning ... ok 2019-12-09T12:17:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock ... ok 2019-12-09T12:17:57.205 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock_invalid_bucket ... ok 2019-12-09T12:18:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention ... ok 2019-12-09T12:18:09.990 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_bucket ... ok 2019-12-09T12:18:18.532 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_mode ... ok 2019-12-09T12:18:29.333 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention ... ok 2019-12-09T12:18:33.931 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention_invalid_bucket ... ok 2019-12-09T12:18:46.277 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_versionid ... ok 2019-12-09T12:18:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_override_default_retention ... ok 2019-12-09T12:19:13.425 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_increase_period ... ok 2019-12-09T12:19:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period ... ok 2019-12-09T12:19:34.881 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period_bypass ... ok 2019-12-09T12:19:44.663 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_retention ... ok 2019-12-09T12:19:52.928 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold ... ok 2019-12-09T12:19:56.512 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_bucket ... ok 2019-12-09T12:20:01.515 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_status ... ok 2019-12-09T12:20:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold ... ok 2019-12-09T12:20:14.979 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold_invalid_bucket ... ok 2019-12-09T12:20:22.413 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_on ... ok 2019-12-09T12:20:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_off ... ok 2019-12-09T12:20:43.204 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_metadata ... ok 2019-12-09T12:20:50.712 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_s3.test_object_lock_uploading_obj ... ok 2019-12-09T12:20:50.737 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.functional.test_utils.test_generate ... ok 2019-12-09T12:21:05.144 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:52:26.603 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.152 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:52:26.612 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.157 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:52:26.617 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.161 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:52:26.621 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.170 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:52:26.653 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.199 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:52:26.682 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.334 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:52:26.793 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.354 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:52:26.814 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:05.371 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:52:26.854 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.464 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:56:28.924 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.470 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:56:28.930 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.473 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:56:28.933 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.480 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:56:28.940 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.514 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:56:28.997 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.520 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:56:29.003 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.589 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:56:29.049 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.639 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:56:29.122 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:25:07.642 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:56:29.103 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.413 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:57:37.873 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.423 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:57:37.883 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.431 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:57:37.891 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.436 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:57:37.918 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.438 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:57:37.898 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.446 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:57:37.929 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.513 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:02:57:37.972 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.541 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:02:57:38.024 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:26:16.542 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:02:57:38.002 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.404 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:02:49.864 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.413 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:02:49.873 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.420 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:02:49.880 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.427 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:02:49.909 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.428 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:02:49.888 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.444 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:02:49.925 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.614 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:02:50.074 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.626 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:02:50.086 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:28.687 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:02:50.169 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.178 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:03:20.638 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.219 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:03:20.679 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.264 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:03:20.723 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.265 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:03:20.748 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.273 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:03:20.756 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.297 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:03:20.755 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.303 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:03:20.763 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.313 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:03:20.796 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:31:59.341 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:03:20.801 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.627 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:04:51.087 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.643 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:04:51.103 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.644 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:04:51.104 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.690 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:04:51.172 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.694 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:04:51.177 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.706 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:04:51.165 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.714 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:04:51.174 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.765 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:04:51.225 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:33:29.792 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:04:51.275 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.592 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:08:00.051 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.595 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:08:00.055 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.613 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:08:00.095 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.619 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:08:00.102 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.622 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:08:00.082 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.627 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:08:00.087 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.726 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:08:00.185 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.761 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:08:00.221 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:36:38.770 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:08:00.253 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.119 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:13:08.579 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.129 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:13:08.587 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.137 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:13:08.620 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.138 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:13:08.598 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.140 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:13:08.623 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.143 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:13:08.603 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.323 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:13:08.805 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.323 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:13:08.783 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:47.389 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:13:08.849 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.132 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:13:12.592 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.159 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:13:12.618 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.175 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:13:12.635 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.190 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:13:12.650 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.196 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:13:12.679 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.197 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:13:12.657 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.200 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:13:12.683 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.212 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:13:12.672 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:41:51.228 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:13:12.711 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:58.927 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:18:20.387 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:58.936 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:18:20.396 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:58.938 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:18:20.398 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:58.942 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:18:20.401 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:58.943 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:18:20.426 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:58.949 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:18:20.432 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:59.140 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:18:20.600 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:59.149 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:18:20.609 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:46:59.173 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:18:20.656 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.694 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:21:39.153 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.696 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:21:39.156 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.738 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:21:39.197 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.742 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:21:39.202 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.778 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:21:39.261 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.781 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:21:39.264 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.834 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:21:39.293 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.864 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:21:39.324 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:17.987 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:21:39.470 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.225 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:23:30.684 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.253 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:23:30.713 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.287 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:23:30.746 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.289 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:23:30.749 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.290 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:23:30.773 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.292 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:23:30.775 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.338 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:23:30.797 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.367 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:23:30.827 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:52:09.385 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:23:30.868 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.215 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:28:44.674 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.241 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:28:44.700 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.261 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:28:44.721 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.262 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:28:44.722 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.278 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:28:44.761 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.279 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:28:44.762 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.442 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:28:44.902 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.457 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:28:44.940 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:57:23.461 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:28:44.921 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.242 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:30:00.701 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.298 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:30:00.758 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.321 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:30:00.804 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.332 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:30:00.815 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.366 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:30:00.826 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.373 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:30:00.833 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.385 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:30:00.846 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.408 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:30:00.891 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T12:58:39.431 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:30:00.891 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:00.915 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:38:22.375 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:00.938 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:38:22.394 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:00.947 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:38:22.431 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:00.952 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:38:22.435 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:00.962 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:38:22.422 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:00.978 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:38:22.438 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:01.183 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:38:22.667 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:01.189 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:38:22.649 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:01.237 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:38:22.697 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.149 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:39:07.609 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.157 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:39:07.617 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.165 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:39:07.625 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.171 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:39:07.654 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.175 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:39:07.635 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.176 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:39:07.659 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.223 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:39:07.682 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.261 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:39:07.719 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:46.261 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:39:07.737 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.302 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:44:18.762 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.309 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:44:18.769 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.325 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:44:18.807 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.325 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:44:18.781 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.328 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:44:18.811 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.329 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:44:18.789 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.505 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:44:18.965 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.531 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:44:18.991 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:12:57.561 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:44:19.044 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.753 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:46:44.213 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.771 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:46:44.231 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.808 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:46:44.291 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.816 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:46:44.299 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.819 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:46:44.279 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.824 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:46:44.284 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.895 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:46:44.355 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.965 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:46:44.425 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:15:22.993 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:46:44.476 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.311 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:49:29.771 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.315 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:49:29.776 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.327 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:49:29.787 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.330 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:49:29.790 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.344 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:49:29.827 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.358 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:49:29.836 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.429 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:49:29.889 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.453 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:49:29.913 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:18:08.517 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:49:30.000 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.082 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:54:43.542 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.092 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:54:43.552 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.101 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:54:43.584 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.104 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:54:43.564 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.106 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:54:43.589 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.107 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:54:43.567 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.309 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:54:43.766 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.325 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:54:43.807 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:22.369 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:54:43.829 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.248 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:55:05.701 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.278 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:55:05.738 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.299 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:55:05.782 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.302 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:55:05.785 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.320 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:55:05.781 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.336 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:55:05.819 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.344 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:55:05.805 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.362 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:55:05.822 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:23:44.385 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:55:05.845 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.246 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:59:55.706 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.258 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:59:55.718 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.263 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:59:55.747 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.265 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:59:55.726 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.269 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:59:55.752 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.270 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:59:55.730 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.434 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:03:59:55.917 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.450 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:03:59:55.910 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:34.462 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:03:59:55.922 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:05.861 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:03:27.322 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:05.863 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:03:27.323 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:05.948 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:03:27.431 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:05.950 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:03:27.433 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:05.991 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:03:27.440 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:05.991 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:03:27.445 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:06.002 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:03:27.462 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:06.089 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:03:27.566 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:06.089 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:03:27.541 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.409 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:05:05.868 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.413 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:05:05.873 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.489 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:05:05.949 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.494 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:05:05.954 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.495 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:05:05.956 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.572 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:05:06.032 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.744 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:05:06.227 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.748 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:05:06.231 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:44.823 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:05:06.306 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.635 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:10:18.095 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.644 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:10:18.104 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.650 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:10:18.110 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.653 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:10:18.135 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.654 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:10:18.114 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.656 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:10:18.139 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.846 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:10:18.307 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.859 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:10:18.342 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:56.866 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:10:18.326 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.479 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:11:48.939 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.488 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:11:48.941 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.509 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:11:48.969 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.538 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:11:48.997 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.539 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:11:48.998 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.554 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:11:49.036 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.557 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:11:49.040 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.587 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:11:49.047 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:40:27.626 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:11:49.109 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.238 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:15:29.698 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.251 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:15:29.712 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.259 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:15:29.719 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.262 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:15:29.745 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.266 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:15:29.726 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.270 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:15:29.753 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.394 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:15:29.853 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.418 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:15:29.878 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:08.458 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:15:29.941 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T13:48:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T13:48:02.616 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T13:48:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T13:48:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T13:48:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T13:48:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T13:48:03.000 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T13:48:03.171 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T13:48:03.665 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T13:48:03.717 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T13:48:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T13:48:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T13:48:03.748 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T13:48:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T13:48:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T13:48:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T13:48:03.749 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T13:48:03.750 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T13:48:03.750 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T13:48:03.751 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T13:48:03.751 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T13:48:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T13:48:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T13:48:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T13:48:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T13:48:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T13:48:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T13:48:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T13:48:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T13:48:03.803 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T13:48:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T13:48:03.838 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T13:48:04.105 INFO:teuthology.orchestra.run.smithi033.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T13:48:04.105 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T13:48:04.106 INFO:teuthology.orchestra.run.smithi033.stderr:---------------------------------------------------------------------- 2019-12-09T13:48:04.106 INFO:teuthology.orchestra.run.smithi033.stderr:Ran 611 tests in 11709.828s 2019-12-09T13:48:04.106 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T13:48:04.106 INFO:teuthology.orchestra.run.smithi033.stderr:OK (SKIP=75) 2019-12-09T13:48:04.164 DEBUG:tasks.s3tests:Scanning radosgw logs for leaked encryption keys... 2019-12-09T13:48:04.164 INFO:teuthology.orchestra.run.smithi033:> grep --binary-files=text pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= /var/log/ceph/rgw.ceph.client.0.log 2019-12-09T13:48:04.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T13:48:04.292 INFO:tasks.s3tests:Cleaning up boto... 2019-12-09T13:48:04.292 INFO:teuthology.orchestra.run.smithi033:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T13:48:04.340 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2019-12-09T13:48:04.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.450+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.492 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.488+0000 7f88be83aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:04.492 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.488+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.499+0000 7f88be83aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:04.503 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.499+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.511 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.504+0000 7f88be83aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:04.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.504+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.524 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.518+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.656+0000 7f88be83aec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T13:48:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.656+0000 7f88be83aec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T13:48:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.656+0000 7f88be83aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:48:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.656+0000 7f88be83aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:48:04.661 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.656+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.670 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.666+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:04.670 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.666+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.678 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.674+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:04.678 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.674+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.682+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:04.686 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.682+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.692+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:04.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.692+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.700+0000 7f88be83aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.700+0000 7f88be83aec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:48:04.704 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.700+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.709+0000 7f88be83aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:04.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.709+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.718+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.718+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.732+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:04.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.732+0000 7f88be83aec0 20 zonegroup default 2019-12-09T13:48:04.736 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.732+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.737+0000 7f88be83aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:04.741 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.737+0000 7f88be83aec0 10 Cannot find current period zone using local zone 2019-12-09T13:48:04.742 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.737+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.744+0000 7f88be83aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.744+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.753+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.753+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.764+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.764+0000 7f88be83aec0 20 zone default found 2019-12-09T13:48:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.764+0000 7f88be83aec0 20 started sync module instance, tier type = 2019-12-09T13:48:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:04.764+0000 7f88be83aec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T13:48:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.275+0000 7f88be83aec0 20 add_watcher() i=0 2019-12-09T13:48:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.275+0000 7f88be83aec0 20 add_watcher() i=1 2019-12-09T13:48:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.275+0000 7f88be83aec0 20 add_watcher() i=2 2019-12-09T13:48:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.275+0000 7f88be83aec0 20 add_watcher() i=3 2019-12-09T13:48:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.275+0000 7f88be83aec0 20 add_watcher() i=4 2019-12-09T13:48:05.279 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.275+0000 7f88be83aec0 20 add_watcher() i=5 2019-12-09T13:48:05.280 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.275+0000 7f88be83aec0 20 add_watcher() i=6 2019-12-09T13:48:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.309+0000 7f88be83aec0 20 add_watcher() i=7 2019-12-09T13:48:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.309+0000 7f88be83aec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:48:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.309+0000 7f87fffff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:48:05.317 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.314+0000 7f88ab263700 0 -- 172.21.15.33:0/979842181 >> v2:172.21.15.186:3300/0 conn(0x56528fab26c0 msgr2=0x56528fab4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:48:05.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:05.367+0000 7f88aba64700 0 -- 172.21.15.33:0/1209313587 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x56528fac3880 msgr2=0x56528fac5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:07.319 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.314+0000 7f87ea7fc700 20 reqs_thread_entry: start 2019-12-09T13:48:07.319 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.315+0000 7f87e3fff700 20 reqs_thread_entry: start 2019-12-09T13:48:07.320 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.315+0000 7f88be83aec0 20 init_complete bucket index max shards: 0 2019-12-09T13:48:07.320 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.316+0000 7f87e27fc700 20 reqs_thread_entry: start 2019-12-09T13:48:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.316+0000 7f88be83aec0 20 get_system_obj_state: rctx=0x7ffef896c768 obj=default.rgw.meta:users.uid:bar.client.0 state=0x56528fb2d010 s->prefetch_data=0 2019-12-09T13:48:07.321 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.316+0000 7f88be83aec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T13:48:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.339+0000 7f88be83aec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T13:48:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.339+0000 7f88be83aec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:48:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.339+0000 7f88be83aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:48:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.339+0000 7f88be83aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:48:07.344 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.339+0000 7f88be83aec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:48:07.345 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.339+0000 7f88be83aec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:07.371 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.367+0000 7f88be83aec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T13:48:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.367+0000 7f88be83aec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T13:48:07.372 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.367+0000 7f88be83aec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:48:07.390 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.386+0000 7f88be83aec0 10 removing key index: ZHBTIXTHSUXLSWRSDXEF 2019-12-09T13:48:07.391 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.386+0000 7f88be83aec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=170 2019-12-09T13:48:07.415 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.406+0000 7f88917fa700 10 RGWWatcher::handle_notify() notify_id 592705486997 cookie 94912597582160 notifier 4649 bl.length()=170 2019-12-09T13:48:07.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.776+0000 7f88be83aec0 10 removing email index: bar.client.0+test@test.test 2019-12-09T13:48:07.780 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.776+0000 7f88be83aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=178 2019-12-09T13:48:07.797 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:07.792+0000 7f88917fa700 10 RGWWatcher::handle_notify() notify_id 592705486998 cookie 94912597570976 notifier 4649 bl.length()=178 2019-12-09T13:48:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:08.401+0000 7f88be83aec0 10 removing user buckets index 2019-12-09T13:48:08.405 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:08.401+0000 7f88be83aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=169 2019-12-09T13:48:08.417 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:08.413+0000 7f88917fa700 10 RGWWatcher::handle_notify() notify_id 592705486944 cookie 94912597575536 notifier 4649 bl.length()=169 2019-12-09T13:48:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:08.858+0000 7f88be83aec0 10 removing user index: bar.client.0 2019-12-09T13:48:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:08.858+0000 7f88be83aec0 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2019-12-09T13:48:08.863 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:08.859+0000 7f88be83aec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=161 2019-12-09T13:48:08.877 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:08.873+0000 7f88917fa700 10 RGWWatcher::handle_notify() notify_id 592705486999 cookie 94912597570976 notifier 4649 bl.length()=161 2019-12-09T13:48:09.057 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.052+0000 7f88be83aec0 20 remove_watcher() i=0 2019-12-09T13:48:09.057 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.052+0000 7f88be83aec0 2 removed watcher, disabling cache 2019-12-09T13:48:09.093 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.089+0000 7f88be83aec0 20 remove_watcher() i=1 2019-12-09T13:48:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.147+0000 7f88be83aec0 20 remove_watcher() i=2 2019-12-09T13:48:09.201 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.196+0000 7f88be83aec0 20 remove_watcher() i=3 2019-12-09T13:48:09.266 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.261+0000 7f88be83aec0 20 remove_watcher() i=4 2019-12-09T13:48:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.318+0000 7f88be83aec0 20 remove_watcher() i=5 2019-12-09T13:48:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.380+0000 7f88be83aec0 20 remove_watcher() i=6 2019-12-09T13:48:09.449 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.444+0000 7f88be83aec0 20 remove_watcher() i=7 2019-12-09T13:48:09.459 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2019-12-09T13:48:09.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.599+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.643 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.639+0000 7faa72c02ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:09.644 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.639+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.652 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.648+0000 7faa72c02ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:09.652 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.648+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.663 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.658+0000 7faa72c02ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:09.663 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.658+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.681 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.676+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:09.880 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.876+0000 7faa72c02ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T13:48:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.876+0000 7faa72c02ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T13:48:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.876+0000 7faa72c02ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:48:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.876+0000 7faa72c02ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:48:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.876+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.894 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.890+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:09.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.890+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.908 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.904+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:09.909 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.904+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.922+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:09.926 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.922+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.940+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:09.945 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.940+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.949+0000 7faa72c02ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.949+0000 7faa72c02ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:48:09.953 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.949+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.959+0000 7faa72c02ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:09.964 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.960+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.976+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:09.981 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.976+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:09.995 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.991+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.991+0000 7faa72c02ec0 20 zonegroup default 2019-12-09T13:48:09.996 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:09.991+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.000+0000 7faa72c02ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.000+0000 7faa72c02ec0 10 Cannot find current period zone using local zone 2019-12-09T13:48:10.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.000+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:10.013 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.008+0000 7faa72c02ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:10.013 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.009+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.028+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:10.033 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.028+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:10.053 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.049+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.049+0000 7faa72c02ec0 20 zone default found 2019-12-09T13:48:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.049+0000 7faa72c02ec0 20 started sync module instance, tier type = 2019-12-09T13:48:10.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.049+0000 7faa72c02ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.445+0000 7faa72c02ec0 20 add_watcher() i=0 2019-12-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.445+0000 7faa72c02ec0 20 add_watcher() i=1 2019-12-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.445+0000 7faa72c02ec0 20 add_watcher() i=2 2019-12-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.445+0000 7faa72c02ec0 20 add_watcher() i=3 2019-12-09T13:48:10.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.445+0000 7faa72c02ec0 20 add_watcher() i=4 2019-12-09T13:48:10.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.445+0000 7faa72c02ec0 20 add_watcher() i=5 2019-12-09T13:48:10.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.447+0000 7faa72c02ec0 20 add_watcher() i=6 2019-12-09T13:48:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.473+0000 7faa72c02ec0 20 add_watcher() i=7 2019-12-09T13:48:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.473+0000 7faa72c02ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:48:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.473+0000 7fa9b3fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:48:10.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.474+0000 7faa6062d700 0 -- 172.21.15.33:0/1362589830 >> v2:172.21.15.33:3300/0 conn(0x55bf661353e0 msgr2=0x55bf661357f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:48:10.518 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:10.513+0000 7faa6062d700 0 -- 172.21.15.33:0/2111392221 >> [v2:172.21.15.33:6800/9135,v1:172.21.15.33:6801/9135] conn(0x7fa9a0022490 msgr2=0x7fa9a0024980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T13:48:12.014 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.011+0000 7fa99e7fc700 20 reqs_thread_entry: start 2019-12-09T13:48:12.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.011+0000 7fa997fff700 20 reqs_thread_entry: start 2019-12-09T13:48:12.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.011+0000 7faa72c02ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:48:12.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.012+0000 7fa9967fc700 20 reqs_thread_entry: start 2019-12-09T13:48:12.015 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.012+0000 7faa72c02ec0 20 get_system_obj_state: rctx=0x7ffc6caa90f8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55bf661b8550 s->prefetch_data=0 2019-12-09T13:48:12.016 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.012+0000 7faa72c02ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T13:48:12.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.028+0000 7faa72c02ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T13:48:12.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.028+0000 7faa72c02ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:48:12.031 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.028+0000 7faa72c02ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:48:12.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.028+0000 7faa72c02ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:48:12.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.028+0000 7faa72c02ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:48:12.032 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.028+0000 7faa72c02ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:12.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.052+0000 7faa72c02ec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T13:48:12.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.052+0000 7faa72c02ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T13:48:12.056 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.052+0000 7faa72c02ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:48:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.077+0000 7faa72c02ec0 10 removing key index: JUOQDBLDEIOWAFNVUCCM 2019-12-09T13:48:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.077+0000 7faa72c02ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=170 2019-12-09T13:48:12.091 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.088+0000 7faa457fa700 10 RGWWatcher::handle_notify() notify_id 592705486945 cookie 94280539595744 notifier 5058 bl.length()=170 2019-12-09T13:48:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.798+0000 7faa72c02ec0 10 removing email index: foo.client.0+test@test.test 2019-12-09T13:48:12.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.798+0000 7faa72c02ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=178 2019-12-09T13:48:12.818 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:12.815+0000 7faa457fa700 10 RGWWatcher::handle_notify() notify_id 592705486902 cookie 94280539605712 notifier 5058 bl.length()=178 2019-12-09T13:48:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:13.275+0000 7faa72c02ec0 10 removing user buckets index 2019-12-09T13:48:13.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:13.275+0000 7faa72c02ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=169 2019-12-09T13:48:13.316 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:13.313+0000 7faa457fa700 10 RGWWatcher::handle_notify() notify_id 592705487000 cookie 94280539593872 notifier 5058 bl.length()=169 2019-12-09T13:48:13.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:13.800+0000 7faa72c02ec0 10 removing user index: foo.client.0 2019-12-09T13:48:13.803 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:13.800+0000 7faa72c02ec0 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2019-12-09T13:48:13.804 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:13.800+0000 7faa72c02ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=161 2019-12-09T13:48:13.820 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:13.817+0000 7faa457fa700 10 RGWWatcher::handle_notify() notify_id 592705487772 cookie 94280539590496 notifier 5058 bl.length()=161 2019-12-09T13:48:14.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.072+0000 7faa72c02ec0 20 remove_watcher() i=0 2019-12-09T13:48:14.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.072+0000 7faa72c02ec0 2 removed watcher, disabling cache 2019-12-09T13:48:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.140+0000 7faa72c02ec0 20 remove_watcher() i=1 2019-12-09T13:48:14.185 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.182+0000 7faa72c02ec0 20 remove_watcher() i=2 2019-12-09T13:48:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.224+0000 7faa72c02ec0 20 remove_watcher() i=3 2019-12-09T13:48:14.272 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.269+0000 7faa72c02ec0 20 remove_watcher() i=4 2019-12-09T13:48:14.315 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.312+0000 7faa72c02ec0 20 remove_watcher() i=5 2019-12-09T13:48:14.366 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.363+0000 7faa72c02ec0 20 remove_watcher() i=6 2019-12-09T13:48:14.429 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.426+0000 7faa72c02ec0 20 remove_watcher() i=7 2019-12-09T13:48:14.438 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2019-12-09T13:48:14.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.551+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.581+0000 7fbe8de55ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.581+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.590+0000 7fbe8de55ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:14.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.590+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.599+0000 7fbe8de55ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:14.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.599+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.646 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.643+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.828+0000 7fbe8de55ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T13:48:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.828+0000 7fbe8de55ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T13:48:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.828+0000 7fbe8de55ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:48:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.828+0000 7fbe8de55ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:48:14.832 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.828+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.846 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.843+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:14.847 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.843+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.861 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.857+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:14.861 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.857+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.871+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:14.874 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.871+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.893 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.889+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:14.893 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.890+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.905 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.901+0000 7fbe8de55ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:14.905 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.901+0000 7fbe8de55ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:48:14.905 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.901+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.910+0000 7fbe8de55ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:14.913 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.910+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.924 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.921+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:14.925 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.921+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.940+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.940+0000 7fbe8de55ec0 20 zonegroup default 2019-12-09T13:48:14.944 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.940+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.951 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.948+0000 7fbe8de55ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.948+0000 7fbe8de55ec0 10 Cannot find current period zone using local zone 2019-12-09T13:48:14.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.948+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.956+0000 7fbe8de55ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:14.961 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.956+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.973 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.970+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:14.974 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.970+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.979+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.979+0000 7fbe8de55ec0 20 zone default found 2019-12-09T13:48:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.979+0000 7fbe8de55ec0 20 started sync module instance, tier type = 2019-12-09T13:48:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:14.979+0000 7fbe8de55ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T13:48:15.346 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.343+0000 7fbe8de55ec0 20 add_watcher() i=0 2019-12-09T13:48:15.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.343+0000 7fbe8de55ec0 20 add_watcher() i=1 2019-12-09T13:48:15.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.343+0000 7fbe8de55ec0 20 add_watcher() i=2 2019-12-09T13:48:15.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.343+0000 7fbe8de55ec0 20 add_watcher() i=3 2019-12-09T13:48:15.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.343+0000 7fbe8de55ec0 20 add_watcher() i=4 2019-12-09T13:48:15.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.343+0000 7fbe8de55ec0 20 add_watcher() i=5 2019-12-09T13:48:15.347 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.343+0000 7fbe8de55ec0 20 add_watcher() i=6 2019-12-09T13:48:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.386+0000 7fbe8de55ec0 20 add_watcher() i=7 2019-12-09T13:48:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.386+0000 7fbe8de55ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:48:15.390 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.387+0000 7fbe4bfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:48:15.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.389+0000 7fbe7b07f700 0 -- 172.21.15.33:0/1509881518 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x558cc448e3c0 msgr2=0x558cc4490820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:48:15.432 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:15.429+0000 7fbe7a87e700 0 -- 172.21.15.33:0/371608946 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x558cc44a3470 msgr2=0x558cc44a5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:17.223 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.219+0000 7fbdba7fc700 20 reqs_thread_entry: start 2019-12-09T13:48:17.223 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.220+0000 7fbdb3fff700 20 reqs_thread_entry: start 2019-12-09T13:48:17.223 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.220+0000 7fbe8de55ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:48:17.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.220+0000 7fbdb27fc700 20 reqs_thread_entry: start 2019-12-09T13:48:17.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.220+0000 7fbe8de55ec0 20 get_system_obj_state: rctx=0x7ffee2c73298 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x558cc450cc40 s->prefetch_data=0 2019-12-09T13:48:17.224 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.220+0000 7fbe8de55ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T13:48:17.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.238+0000 7fbe8de55ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T13:48:17.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.238+0000 7fbe8de55ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:48:17.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.238+0000 7fbe8de55ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:48:17.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.238+0000 7fbe8de55ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:48:17.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.238+0000 7fbe8de55ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:48:17.242 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.238+0000 7fbe8de55ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:17.257 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.254+0000 7fbe8de55ec0 20 rados_obj.operate() r=0 bl.length=505 2019-12-09T13:48:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.254+0000 7fbe8de55ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T13:48:17.258 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.254+0000 7fbe8de55ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:48:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.261+0000 7fbe8de55ec0 10 removing key index: UDTLXQDBXLENGHOOTUQO 2019-12-09T13:48:17.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.261+0000 7fbe8de55ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=170 2019-12-09T13:48:17.277 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.274+0000 7fbe70ff9700 10 RGWWatcher::handle_notify() notify_id 592705487001 cookie 94063076815136 notifier 5076 bl.length()=170 2019-12-09T13:48:17.680 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.676+0000 7fbe8de55ec0 10 removing email index: testx$tenanteduser.client.0+test@test.test 2019-12-09T13:48:17.680 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.676+0000 7fbe8de55ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=193 2019-12-09T13:48:17.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:17.689+0000 7fbe70ff9700 10 RGWWatcher::handle_notify() notify_id 592705487773 cookie 94063076830256 notifier 5076 bl.length()=193 2019-12-09T13:48:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.005+0000 7fbe8de55ec0 10 removing user buckets index 2019-12-09T13:48:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.005+0000 7fbe8de55ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=184 2019-12-09T13:48:18.035 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.024+0000 7fbe70ff9700 10 RGWWatcher::handle_notify() notify_id 592705486903 cookie 94063076828384 notifier 5076 bl.length()=184 2019-12-09T13:48:18.521 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.518+0000 7fbe8de55ec0 10 removing user index: testx$tenanteduser.client.0 2019-12-09T13:48:18.521 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.518+0000 7fbe8de55ec0 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2019-12-09T13:48:18.522 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.518+0000 7fbe8de55ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=176 2019-12-09T13:48:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.534+0000 7fbe70ff9700 10 RGWWatcher::handle_notify() notify_id 592705487002 cookie 94063076826448 notifier 5076 bl.length()=176 2019-12-09T13:48:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.789+0000 7fbe8de55ec0 20 remove_watcher() i=0 2019-12-09T13:48:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.789+0000 7fbe8de55ec0 2 removed watcher, disabling cache 2019-12-09T13:48:18.845 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.842+0000 7fbe8de55ec0 20 remove_watcher() i=1 2019-12-09T13:48:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.894+0000 7fbe8de55ec0 20 remove_watcher() i=2 2019-12-09T13:48:18.936 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.933+0000 7fbe8de55ec0 20 remove_watcher() i=3 2019-12-09T13:48:18.970 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:18.967+0000 7fbe8de55ec0 20 remove_watcher() i=4 2019-12-09T13:48:19.005 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.002+0000 7fbe8de55ec0 20 remove_watcher() i=5 2019-12-09T13:48:19.038 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.035+0000 7fbe8de55ec0 20 remove_watcher() i=6 2019-12-09T13:48:19.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.068+0000 7fbe8de55ec0 20 remove_watcher() i=7 2019-12-09T13:48:19.086 INFO:tasks.s3tests:Removing s3-tests... 2019-12-09T13:48:19.086 INFO:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/s3-tests 2019-12-09T13:48:19.213 INFO:teuthology.run_tasks:Running task swift... 2019-12-09T13:48:19.234 INFO:tasks.swift:clients=['client.0'] 2019-12-09T13:48:19.234 INFO:tasks.swift:Downloading swift... 2019-12-09T13:48:19.234 INFO:teuthology.orchestra.run.smithi033:> git clone -b ceph-master git://git.ceph.com/git/swift.git /home/ubuntu/cephtest/swift 2019-12-09T13:48:19.352 INFO:teuthology.orchestra.run.smithi033.stdout:Cloning into '/home/ubuntu/cephtest/swift'... 2019-12-09T13:48:19.681 INFO:tasks.swift:Creating rgw users... 2019-12-09T13:48:19.681 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser foo.client.0:foo --display-name 'Mr. foo.client.0 foo' --secret xzNicEEe0LYwNEPphjXzFbarYjKC9Htv1oiZ3iSLlbiOw6DSYSDBEg== --email foo.client.0+test@test.test --key-type swift --access full 2019-12-09T13:48:19.809 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.806+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:19.847 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.843+0000 7feffbc47ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:19.847 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.843+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.851+0000 7feffbc47ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:19.854 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.851+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.861+0000 7feffbc47ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:19.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.861+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:19.889+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:20.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.116+0000 7feffbc47ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T13:48:20.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.116+0000 7feffbc47ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T13:48:20.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.116+0000 7feffbc47ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:48:20.119 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.116+0000 7feffbc47ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:48:20.120 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.116+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.141 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.138+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:20.142 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.139+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.151+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:20.154 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.151+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.166+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:20.169 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.166+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.189+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:20.192 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.189+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.199+0000 7feffbc47ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:20.202 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.199+0000 7feffbc47ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:48:20.203 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.199+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.206+0000 7feffbc47ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:20.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.206+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.218+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:20.221 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.218+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.231+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:20.234 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.231+0000 7feffbc47ec0 20 zonegroup default 2019-12-09T13:48:20.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.231+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.238+0000 7feffbc47ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:20.241 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.238+0000 7feffbc47ec0 10 Cannot find current period zone using local zone 2019-12-09T13:48:20.242 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.238+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.255 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.245+0000 7feffbc47ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:20.256 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.245+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.261+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:20.264 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.261+0000 7feffbc47ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.278+0000 7feffbc47ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.278+0000 7feffbc47ec0 20 zone default found 2019-12-09T13:48:20.281 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.278+0000 7feffbc47ec0 20 started sync module instance, tier type = 2019-12-09T13:48:20.282 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.278+0000 7feffbc47ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T13:48:20.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.581+0000 7fefe8e71700 0 -- 172.21.15.33:0/2438881973 >> [v2:172.21.15.33:6810/10162,v1:172.21.15.33:6811/10162] conn(0x7fefe4008740 msgr2=0x7fefe4008b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.633+0000 7feffbc47ec0 20 add_watcher() i=0 2019-12-09T13:48:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.633+0000 7feffbc47ec0 20 add_watcher() i=1 2019-12-09T13:48:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.633+0000 7feffbc47ec0 20 add_watcher() i=2 2019-12-09T13:48:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.633+0000 7feffbc47ec0 20 add_watcher() i=3 2019-12-09T13:48:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.633+0000 7feffbc47ec0 20 add_watcher() i=4 2019-12-09T13:48:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.633+0000 7feffbc47ec0 20 add_watcher() i=5 2019-12-09T13:48:20.637 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.633+0000 7feffbc47ec0 20 add_watcher() i=6 2019-12-09T13:48:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.664+0000 7feffbc47ec0 20 add_watcher() i=7 2019-12-09T13:48:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.664+0000 7feffbc47ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:48:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.664+0000 7fefb8fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:48:20.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.689+0000 7fefe3fff700 0 -- 172.21.15.33:0/2306268207 >> v2:172.21.15.186:3300/0 conn(0x557e04b966e0 msgr2=0x557e04b98b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:20.892+0000 7fefe3fff700 0 --2- 172.21.15.33:0/2306268207 >> v2:172.21.15.186:3300/0 conn(0x557e04b966e0 0x557e04b98b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T13:48:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.424+0000 7fef2affd700 20 reqs_thread_entry: start 2019-12-09T13:48:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.424+0000 7fef28ff9700 20 reqs_thread_entry: start 2019-12-09T13:48:22.428 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.424+0000 7feffbc47ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.425+0000 7fef1effd700 20 reqs_thread_entry: start 2019-12-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.425+0000 7feffbc47ec0 20 get_system_obj_state: rctx=0x7fffde373f48 obj=default.rgw.meta:users.uid:foo.client.0 state=0x557e04c0acd0 s->prefetch_data=0 2019-12-09T13:48:22.429 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.425+0000 7feffbc47ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.431+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.431+0000 7feffbc47ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.431+0000 7feffbc47ec0 20 get_system_obj_state: rctx=0x7fffde373f58 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x557e04c0ca60 s->prefetch_data=0 2019-12-09T13:48:22.434 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.431+0000 7feffbc47ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T13:48:22.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.437+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T13:48:22.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.437+0000 7feffbc47ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:48:22.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.437+0000 7feffbc47ec0 20 get_system_obj_state: rctx=0x7fffde373b68 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x557e04c0ca60 s->prefetch_data=0 2019-12-09T13:48:22.440 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.437+0000 7feffbc47ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : miss 2019-12-09T13:48:22.447 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.444+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T13:48:22.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.444+0000 7feffbc47ec0 10 adding default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:48:22.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.444+0000 7feffbc47ec0 20 get_system_obj_state: rctx=0x7fffde373ce8 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x557e04c0cd10 s->prefetch_data=0 2019-12-09T13:48:22.448 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.444+0000 7feffbc47ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : type miss (requested=0x6, cached=0x0) 2019-12-09T13:48:22.453 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.450+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T13:48:22.454 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.450+0000 7feffbc47ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:48:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.497+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:48:22.500 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.497+0000 7feffbc47ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:48:22.501 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.497+0000 7feffbc47ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=612 2019-12-09T13:48:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.505+0000 7fefd1ffb700 10 RGWWatcher::handle_notify() notify_id 592705487774 cookie 93999733399584 notifier 4676 bl.length()=612 2019-12-09T13:48:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.505+0000 7fefd1ffb700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:48:22.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.505+0000 7fefd1ffb700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:48:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.800+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T13:48:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.800+0000 7feffbc47ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:48:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.800+0000 7feffbc47ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T13:48:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.811+0000 7fefd1ffb700 10 RGWWatcher::handle_notify() notify_id 592705486904 cookie 93999733414672 notifier 4676 bl.length()=194 2019-12-09T13:48:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.811+0000 7fefd1ffb700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T13:48:22.815 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:22.811+0000 7fefd1ffb700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:48:23.196 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.193+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T13:48:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.193+0000 7feffbc47ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:48:23.197 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.193+0000 7feffbc47ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=183 2019-12-09T13:48:23.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.206+0000 7fefd1ffb700 10 RGWWatcher::handle_notify() notify_id 592705487003 cookie 93999733402944 notifier 4676 bl.length()=183 2019-12-09T13:48:23.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.206+0000 7fefd1ffb700 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T13:48:23.209 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.206+0000 7fefd1ffb700 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T13:48:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.492+0000 7feffbc47ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:48:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.492+0000 7feffbc47ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:48:23.496 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.492+0000 7feffbc47ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=636 2019-12-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.505+0000 7fefd1ffb700 10 RGWWatcher::handle_notify() notify_id 592705487775 cookie 93999733399584 notifier 4676 bl.length()=636 2019-12-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.505+0000 7fefd1ffb700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:48:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.505+0000 7fefd1ffb700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:48:23.880 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T13:48:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "foo.client.0", 2019-12-09T13:48:23.880 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Mr. foo.client.0 foo", 2019-12-09T13:48:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T13:48:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T13:48:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T13:48:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [ 2019-12-09T13:48:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T13:48:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "id": "foo.client.0:foo", 2019-12-09T13:48:23.881 INFO:teuthology.orchestra.run.smithi033.stdout: "permissions": "full-control" 2019-12-09T13:48:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T13:48:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T13:48:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [], 2019-12-09T13:48:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [ 2019-12-09T13:48:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T13:48:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "foo.client.0:foo", 2019-12-09T13:48:23.882 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "xzNicEEe0LYwNEPphjXzFbarYjKC9Htv1oiZ3iSLlbiOw6DSYSDBEg==" 2019-12-09T13:48:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T13:48:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T13:48:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T13:48:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T13:48:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T13:48:23.883 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T13:48:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T13:48:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T13:48:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T13:48:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T13:48:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T13:48:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T13:48:23.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T13:48:23.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T13:48:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T13:48:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T13:48:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T13:48:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T13:48:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T13:48:23.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T13:48:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T13:48:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T13:48:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T13:48:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T13:48:23.888 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T13:48:23.888 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T13:48:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.931+0000 7feffbc47ec0 20 remove_watcher() i=0 2019-12-09T13:48:23.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:23.931+0000 7feffbc47ec0 2 removed watcher, disabling cache 2019-12-09T13:48:24.003 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.000+0000 7feffbc47ec0 20 remove_watcher() i=1 2019-12-09T13:48:24.068 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.065+0000 7feffbc47ec0 20 remove_watcher() i=2 2019-12-09T13:48:24.140 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.136+0000 7feffbc47ec0 20 remove_watcher() i=3 2019-12-09T13:48:24.200 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.197+0000 7feffbc47ec0 20 remove_watcher() i=4 2019-12-09T13:48:24.278 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.275+0000 7feffbc47ec0 20 remove_watcher() i=5 2019-12-09T13:48:24.333 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.329+0000 7feffbc47ec0 20 remove_watcher() i=6 2019-12-09T13:48:24.391 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.387+0000 7feffbc47ec0 20 remove_watcher() i=7 2019-12-09T13:48:24.398 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser bar.client.0:bar --display-name 'Mr. bar.client.0 bar' --secret Qz7hFOhkRNqtlUwRDoRG+brPXyzl6rIY0k/Yu6+U64bBBuMfmhR23g== --email bar.client.0+test@test.test --key-type swift --access full 2019-12-09T13:48:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.514+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.544 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.540+0000 7f2d740a4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:24.544 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.540+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.548+0000 7f2d740a4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:24.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.548+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.557+0000 7f2d740a4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:24.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.557+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.573+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.742+0000 7f2d740a4ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T13:48:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.742+0000 7f2d740a4ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T13:48:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.742+0000 7f2d740a4ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:48:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.742+0000 7f2d740a4ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:48:24.745 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.742+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.760+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:24.764 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.760+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.768+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:24.771 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.768+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.783 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.780+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:24.783 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.780+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.798+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:24.801 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.798+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.804+0000 7f2d740a4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:24.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.804+0000 7f2d740a4ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:48:24.807 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.804+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.810+0000 7f2d740a4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:24.813 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.810+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.824 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.821+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:24.825 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.821+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.834+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:24.838 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.834+0000 7f2d740a4ec0 20 zonegroup default 2019-12-09T13:48:24.838 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.834+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.840+0000 7f2d740a4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:24.843 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.840+0000 7f2d740a4ec0 10 Cannot find current period zone using local zone 2019-12-09T13:48:24.844 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.840+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.852 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.849+0000 7f2d740a4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:24.852 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.849+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.859+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:24.864 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.859+0000 7f2d740a4ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:24.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.866+0000 7f2d740a4ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:24.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.866+0000 7f2d740a4ec0 20 zone default found 2019-12-09T13:48:24.870 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.866+0000 7f2d740a4ec0 20 started sync module instance, tier type = 2019-12-09T13:48:24.871 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:24.866+0000 7f2d740a4ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T13:48:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.232+0000 7f2d740a4ec0 20 add_watcher() i=0 2019-12-09T13:48:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.232+0000 7f2d740a4ec0 20 add_watcher() i=1 2019-12-09T13:48:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.232+0000 7f2d740a4ec0 20 add_watcher() i=2 2019-12-09T13:48:25.235 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.232+0000 7f2d740a4ec0 20 add_watcher() i=3 2019-12-09T13:48:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.232+0000 7f2d740a4ec0 20 add_watcher() i=4 2019-12-09T13:48:25.236 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.232+0000 7f2d740a4ec0 20 add_watcher() i=5 2019-12-09T13:48:25.238 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.235+0000 7f2d740a4ec0 20 add_watcher() i=6 2019-12-09T13:48:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.268+0000 7f2d740a4ec0 20 add_watcher() i=7 2019-12-09T13:48:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.268+0000 7f2d740a4ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:48:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.268+0000 7f2d31fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:48:25.274 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.271+0000 7f2d612ce700 0 -- 172.21.15.33:0/287325644 >> v1:172.21.15.186:6789/0 conn(0x5636c30e7b90 legacy=0x5636c30ece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:25.318 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:25.315+0000 7f2d60acd700 0 -- 172.21.15.33:0/19472356 >> [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] conn(0x5636c3100c20 msgr2=0x5636c3103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.439+0000 7f2ca3fff700 20 reqs_thread_entry: start 2019-12-09T13:48:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.440+0000 7f2ca1ffb700 20 reqs_thread_entry: start 2019-12-09T13:48:27.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.440+0000 7f2d740a4ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:48:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.440+0000 7f2c8ffff700 20 reqs_thread_entry: start 2019-12-09T13:48:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.440+0000 7f2d740a4ec0 20 get_system_obj_state: rctx=0x7fff3285c0d8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x5636c316a480 s->prefetch_data=0 2019-12-09T13:48:27.444 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.440+0000 7f2d740a4ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T13:48:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.447+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T13:48:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.447+0000 7f2d740a4ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:48:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.447+0000 7f2d740a4ec0 20 get_system_obj_state: rctx=0x7fff3285c0e8 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x5636c316c210 s->prefetch_data=0 2019-12-09T13:48:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.447+0000 7f2d740a4ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T13:48:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.453+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T13:48:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.453+0000 7f2d740a4ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:48:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.453+0000 7f2d740a4ec0 20 get_system_obj_state: rctx=0x7fff3285bcf8 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x5636c316c210 s->prefetch_data=0 2019-12-09T13:48:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.453+0000 7f2d740a4ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : miss 2019-12-09T13:48:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.462+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T13:48:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.462+0000 7f2d740a4ec0 10 adding default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:48:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.462+0000 7f2d740a4ec0 20 get_system_obj_state: rctx=0x7fff3285be78 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x5636c316c4c0 s->prefetch_data=0 2019-12-09T13:48:27.466 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.462+0000 7f2d740a4ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : type miss (requested=0x6, cached=0x0) 2019-12-09T13:48:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.468+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T13:48:27.478 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.469+0000 7f2d740a4ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:48:27.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.508+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:48:27.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.508+0000 7f2d740a4ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:48:27.514 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.508+0000 7f2d740a4ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=612 2019-12-09T13:48:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.519+0000 7f2d427fc700 10 RGWWatcher::handle_notify() notify_id 592705487004 cookie 94793200662304 notifier 5103 bl.length()=612 2019-12-09T13:48:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.519+0000 7f2d427fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:48:27.523 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.519+0000 7f2d427fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:48:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.832+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T13:48:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.832+0000 7f2d740a4ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:48:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.832+0000 7f2d740a4ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T13:48:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.848+0000 7f2d427fc700 10 RGWWatcher::handle_notify() notify_id 592705487005 cookie 94793200662304 notifier 5103 bl.length()=194 2019-12-09T13:48:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.848+0000 7f2d427fc700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T13:48:27.851 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:27.848+0000 7f2d427fc700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:48:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.433+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T13:48:28.436 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.433+0000 7f2d740a4ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:48:28.437 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.433+0000 7f2d740a4ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=183 2019-12-09T13:48:28.452 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.449+0000 7f2d427fc700 10 RGWWatcher::handle_notify() notify_id 592705487006 cookie 94793200672784 notifier 5103 bl.length()=183 2019-12-09T13:48:28.453 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.449+0000 7f2d427fc700 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T13:48:28.453 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.449+0000 7f2d427fc700 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T13:48:28.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.710+0000 7f2d740a4ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:48:28.713 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.710+0000 7f2d740a4ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:48:28.714 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.710+0000 7f2d740a4ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=636 2019-12-09T13:48:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.727+0000 7f2d427fc700 10 RGWWatcher::handle_notify() notify_id 592705487007 cookie 94793200662304 notifier 5103 bl.length()=636 2019-12-09T13:48:28.730 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.727+0000 7f2d427fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:48:28.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.727+0000 7f2d427fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:48:28.924 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2019-12-09T13:48:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "user_id": "bar.client.0", 2019-12-09T13:48:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "display_name": "Mr. bar.client.0 bar", 2019-12-09T13:48:28.924 INFO:teuthology.orchestra.run.smithi033.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T13:48:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "suspended": 0, 2019-12-09T13:48:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max_buckets": 1000, 2019-12-09T13:48:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "subusers": [ 2019-12-09T13:48:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T13:48:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "id": "bar.client.0:bar", 2019-12-09T13:48:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: "permissions": "full-control" 2019-12-09T13:48:28.925 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T13:48:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T13:48:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "keys": [], 2019-12-09T13:48:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "swift_keys": [ 2019-12-09T13:48:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: { 2019-12-09T13:48:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "user": "bar.client.0:bar", 2019-12-09T13:48:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: "secret_key": "Qz7hFOhkRNqtlUwRDoRG+brPXyzl6rIY0k/Yu6+U64bBBuMfmhR23g==" 2019-12-09T13:48:28.926 INFO:teuthology.orchestra.run.smithi033.stdout: } 2019-12-09T13:48:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2019-12-09T13:48:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": [], 2019-12-09T13:48:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "op_mask": "read, write, delete", 2019-12-09T13:48:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_placement": "", 2019-12-09T13:48:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "default_storage_class": "", 2019-12-09T13:48:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "placement_tags": [], 2019-12-09T13:48:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: "bucket_quota": { 2019-12-09T13:48:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T13:48:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T13:48:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T13:48:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T13:48:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T13:48:28.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "user_quota": { 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": false, 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "check_on_raw": false, 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size": -1, 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max_size_kb": 0, 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max_objects": -1 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2019-12-09T13:48:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: "temp_url_keys": [], 2019-12-09T13:48:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "rgw", 2019-12-09T13:48:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: "mfa_ids": [] 2019-12-09T13:48:28.930 INFO:teuthology.orchestra.run.smithi033.stdout:} 2019-12-09T13:48:28.930 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T13:48:28.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.965+0000 7f2d740a4ec0 20 remove_watcher() i=0 2019-12-09T13:48:28.968 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:28.965+0000 7f2d740a4ec0 2 removed watcher, disabling cache 2019-12-09T13:48:29.025 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:29.022+0000 7f2d740a4ec0 20 remove_watcher() i=1 2019-12-09T13:48:29.075 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:29.072+0000 7f2d740a4ec0 20 remove_watcher() i=2 2019-12-09T13:48:29.120 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:29.117+0000 7f2d740a4ec0 20 remove_watcher() i=3 2019-12-09T13:48:29.161 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:29.158+0000 7f2d740a4ec0 20 remove_watcher() i=4 2019-12-09T13:48:29.210 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:29.207+0000 7f2d740a4ec0 20 remove_watcher() i=5 2019-12-09T13:48:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:29.258+0000 7f2d740a4ec0 20 remove_watcher() i=6 2019-12-09T13:48:29.304 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T13:48:29.300+0000 7f2d740a4ec0 20 remove_watcher() i=7 2019-12-09T13:48:29.312 INFO:tasks.swift:Configuring testswift... 2019-12-09T13:48:29.312 INFO:teuthology.orchestra.run.smithi033:> cd /home/ubuntu/cephtest/swift && ./bootstrap 2019-12-09T13:48:31.913 INFO:teuthology.orchestra.run.smithi033.stdout:New python executable in /home/ubuntu/cephtest/swift/virtualenv/bin/python 2019-12-09T13:48:33.441 INFO:teuthology.orchestra.run.smithi033.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T13:48:33.722 INFO:teuthology.orchestra.run.smithi033.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T13:48:33.833 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting pip 2019-12-09T13:48:33.919 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T13:48:33.988 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: pip 2019-12-09T13:48:33.988 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: pip 9.0.1 2019-12-09T13:48:34.050 INFO:teuthology.orchestra.run.smithi033.stdout: Uninstalling pip-9.0.1: 2019-12-09T13:48:34.094 INFO:teuthology.orchestra.run.smithi033.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T13:48:34.722 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed pip-19.3.1 2019-12-09T13:48:35.059 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T13:48:35.561 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting setuptools==32.3.1 2019-12-09T13:48:35.564 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T13:48:35.606 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: setuptools 2019-12-09T13:48:35.606 INFO:teuthology.orchestra.run.smithi033.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T13:48:35.621 INFO:teuthology.orchestra.run.smithi033.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T13:48:35.629 INFO:teuthology.orchestra.run.smithi033.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T13:48:35.757 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed setuptools-32.3.1 2019-12-09T13:48:36.194 INFO:teuthology.orchestra.run.smithi033.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T13:48:36.350 INFO:teuthology.orchestra.run.smithi033.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T13:48:36.413 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting nose>=1.0.0 2019-12-09T13:48:36.414 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T13:48:36.522 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting boto>=2.0b4 2019-12-09T13:48:36.528 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T13:48:36.679 INFO:teuthology.orchestra.run.smithi033.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T13:48:36.948 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting gevent>=0.14 2019-12-09T13:48:36.962 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:48:37.242 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting isodate>=0.4.4 2019-12-09T13:48:37.244 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl 2019-12-09T13:48:37.311 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting eventlet>=0.9.14 2019-12-09T13:48:37.356 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/ef/01/83454d11bac9015f34e93cf11efcae169390ddf2df97cb73ca86de6465ed/eventlet-0.25.1-py2.py3-none-any.whl (222kB) 2019-12-09T13:48:37.556 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting netifaces>=0.5 2019-12-09T13:48:37.567 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/7e/02/ad1a92a72620cc17d448fe4dbdfbdf8fe1487ee7bfd82bb48308712c2f3c/netifaces-0.10.9-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:48:37.664 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting xattr<0.7.0,>=0.6 2019-12-09T13:48:37.674 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/c5/80/b25d549ae4bf4f3e9635a331b759ffca2de4dd8a78dc5106d1ca92f5d08d/xattr-0.6.4.tar.gz 2019-12-09T13:48:38.150 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting simplejson>=2.1.3 2019-12-09T13:48:38.160 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/98/87/a7b98aa9256c8843f92878966dc3d8d914c14aad97e2c5ce4798d5743e07/simplejson-3.17.0.tar.gz (83kB) 2019-12-09T13:48:38.537 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting WebOb==1.0 2019-12-09T13:48:38.547 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/bc/eb/91b8dd8f3a8d3c3943bf27a2d322501ad8b227bf29f59e792c2eae092bc9/WebOb-1.0.tar.gz (210kB) 2019-12-09T13:48:38.964 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T13:48:38.967 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:48:39.034 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting six 2019-12-09T13:48:39.036 INFO:teuthology.orchestra.run.smithi033.stdout: Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T13:48:39.084 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting monotonic>=1.4 2019-12-09T13:48:39.093 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl 2019-12-09T13:48:39.159 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting dnspython>=1.15.0 2019-12-09T13:48:39.169 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB) 2019-12-09T13:48:39.284 INFO:teuthology.orchestra.run.smithi033.stdout:Collecting enum34; python_version < "3.4" 2019-12-09T13:48:39.291 INFO:teuthology.orchestra.run.smithi033.stdout: Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl 2019-12-09T13:48:39.322 INFO:teuthology.orchestra.run.smithi033.stdout:Building wheels for collected packages: xattr, simplejson, WebOb 2019-12-09T13:48:39.323 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for xattr (setup.py): started 2019-12-09T13:48:39.824 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for xattr (setup.py): finished with status 'done' 2019-12-09T13:48:39.825 INFO:teuthology.orchestra.run.smithi033.stdout: Created wheel for xattr: filename=xattr-0.6.4-cp27-cp27mu-linux_x86_64.whl size=24121 sha256=992815495da0154cf51104f197ab46bba95c75c36b4074eae6fee904f1956e19 2019-12-09T13:48:39.825 INFO:teuthology.orchestra.run.smithi033.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/cb/2c/40/ea48345a21b521cdacebf61db13e3bf0c42c77ec664e3fe9e5 2019-12-09T13:48:39.832 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for simplejson (setup.py): started 2019-12-09T13:48:40.870 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for simplejson (setup.py): finished with status 'done' 2019-12-09T13:48:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: Created wheel for simplejson: filename=simplejson-3.17.0-cp27-cp27mu-linux_x86_64.whl size=113124 sha256=6a34883410b88c3e30c51ddca9a1f430865f77abe89a59bd593b590492bf6e38 2019-12-09T13:48:40.871 INFO:teuthology.orchestra.run.smithi033.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/86/c0/83/dcd0339abb2640544bb8e0938aab2d069cef55e5647ce6e097 2019-12-09T13:48:40.882 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for WebOb (setup.py): started 2019-12-09T13:48:41.120 INFO:teuthology.orchestra.run.smithi033.stdout: Building wheel for WebOb (setup.py): finished with status 'done' 2019-12-09T13:48:41.120 INFO:teuthology.orchestra.run.smithi033.stdout: Created wheel for WebOb: filename=WebOb-1.0-cp27-none-any.whl size=58478 sha256=a79d9b3d638802be7ea5a70cd200dda43d1a390b03a6c2969f8a852cc92f416f 2019-12-09T13:48:41.120 INFO:teuthology.orchestra.run.smithi033.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/30/41/c0/8ff9e679d72e05f355e6e2ec87d026131538c953a3b12496eb 2019-12-09T13:48:41.129 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully built xattr simplejson WebOb 2019-12-09T13:48:41.172 INFO:teuthology.orchestra.run.smithi033.stdout:Installing collected packages: PyYAML, nose, boto, bunch, greenlet, gevent, six, isodate, monotonic, dnspython, enum34, eventlet, netifaces, xattr, simplejson, WebOb 2019-12-09T13:48:42.650 INFO:teuthology.orchestra.run.smithi033.stdout:Successfully installed PyYAML-5.2 WebOb-1.0 boto-2.49.0 bunch-1.0.1 dnspython-1.16.0 enum34-1.1.6 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 isodate-0.6.0 monotonic-1.5 netifaces-0.10.9 nose-1.3.7 simplejson-3.17.0 six-1.13.0 xattr-0.6.4 2019-12-09T13:48:42.968 INFO:teuthology.orchestra.run.smithi033.stdout:running develop 2019-12-09T13:48:43.016 INFO:teuthology.orchestra.run.smithi033.stdout:running egg_info 2019-12-09T13:48:43.016 INFO:teuthology.orchestra.run.smithi033.stdout:creating swift.egg-info 2019-12-09T13:48:43.016 INFO:teuthology.orchestra.run.smithi033.stdout:writing swift.egg-info/PKG-INFO 2019-12-09T13:48:43.016 INFO:teuthology.orchestra.run.smithi033.stdout:writing top-level names to swift.egg-info/top_level.txt 2019-12-09T13:48:43.016 INFO:teuthology.orchestra.run.smithi033.stdout:writing dependency_links to swift.egg-info/dependency_links.txt 2019-12-09T13:48:43.020 INFO:teuthology.orchestra.run.smithi033.stdout:writing entry points to swift.egg-info/entry_points.txt 2019-12-09T13:48:43.021 INFO:teuthology.orchestra.run.smithi033.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T13:48:43.024 INFO:teuthology.orchestra.run.smithi033.stdout:reading manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T13:48:43.025 INFO:teuthology.orchestra.run.smithi033.stdout:reading manifest template 'MANIFEST.in' 2019-12-09T13:48:43.025 INFO:teuthology.orchestra.run.smithi033.stderr:warning: no files found matching 'ChangeLog' 2019-12-09T13:48:43.028 INFO:teuthology.orchestra.run.smithi033.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T13:48:43.030 INFO:teuthology.orchestra.run.smithi033.stdout:running build_ext 2019-12-09T13:48:43.030 INFO:teuthology.orchestra.run.smithi033.stdout:Creating /home/ubuntu/cephtest/swift/virtualenv/lib/python2.7/site-packages/swift.egg-link (link to .) 2019-12-09T13:48:43.030 INFO:teuthology.orchestra.run.smithi033.stdout:Adding swift 1.4.4 to easy-install.pth file 2019-12-09T13:48:43.047 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.047 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-account-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.050 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-account-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.055 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-account-reaper script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.055 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-account-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.055 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-account-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.055 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-container-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.056 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-container-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.056 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-container-sync script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.056 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-container-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.062 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-container-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.062 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-drive-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.065 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-get-nodes script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.065 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-init script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.066 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-object-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.067 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-object-expirer script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.071 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-object-info script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.071 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-object-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.071 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-object-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.071 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-object-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.072 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-proxy-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.072 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-ring-builder script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.073 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-stats-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.079 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-stats-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.081 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-dispersion-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.083 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-dispersion-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.085 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-bench script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.088 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-recon script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.089 INFO:teuthology.orchestra.run.smithi033.stdout:Installing swift-recon-cron script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T13:48:43.090 INFO:teuthology.orchestra.run.smithi033.stdout: 2019-12-09T13:48:43.090 INFO:teuthology.orchestra.run.smithi033.stdout:Installed /home/ubuntu/cephtest/swift 2019-12-09T13:48:43.090 INFO:teuthology.orchestra.run.smithi033.stdout:Processing dependencies for swift==1.4.4 2019-12-09T13:48:43.091 INFO:teuthology.orchestra.run.smithi033.stdout:Finished processing dependencies for swift==1.4.4 2019-12-09T13:48:43.102 INFO:teuthology.orchestra.run.smithi033:> cat > /home/ubuntu/cephtest/archive/testswift.client.0.conf 2019-12-09T13:48:43.146 INFO:teuthology.orchestra.run.smithi033:> SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw' 2019-12-09T13:48:49.084 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:20:10.544 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.102 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:20:10.561 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.112 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:20:10.573 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.142 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:20:10.602 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.152 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:20:10.635 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.154 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:20:10.637 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.263 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:20:10.746 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.310 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:20:10.770 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:48:49.331 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:20:10.791 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:07.000 INFO:teuthology.orchestra.run.smithi033.stderr:testAccountHead (test.functional.tests.TestAccount) ... ok 2019-12-09T13:49:16.292 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerListing (test.functional.tests.TestAccount) ... ok 2019-12-09T13:49:19.309 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:20:40.769 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.323 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:20:40.783 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.330 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:20:40.790 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.332 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:20:40.815 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.337 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:20:40.820 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.338 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:20:40.799 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.369 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:20:40.829 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.393 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:20:40.876 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:49:19.400 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:20:40.860 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ok 2019-12-09T13:52:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ok 2019-12-09T13:52:47.063 INFO:teuthology.orchestra.run.smithi033.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ok 2019-12-09T13:52:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ok 2019-12-09T13:53:13.950 INFO:teuthology.orchestra.run.smithi033.stderr:testListingLimit (test.functional.tests.TestAccount) ... ok 2019-12-09T13:53:40.232 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:25:01.692 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.259 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:25:01.719 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.281 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:25:01.763 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.323 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:25:01.806 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.331 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:25:01.791 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.334 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:25:01.795 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.449 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:25:01.909 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.492 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:25:01.975 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:40.557 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:25:02.017 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:52.182 INFO:teuthology.orchestra.run.smithi033.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok 2019-12-09T13:53:52.254 INFO:teuthology.orchestra.run.smithi033.stderr:testPUT (test.functional.tests.TestAccount) ... ok 2019-12-09T13:54:31.509 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:25:52.968 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.512 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:25:52.971 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.543 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:25:53.025 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.554 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:25:53.037 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.597 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:25:53.057 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.598 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:25:53.057 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.600 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:25:53.060 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.646 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:25:53.129 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:54:31.665 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:25:53.123 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:10.965 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:28:32.425 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:10.968 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:28:32.429 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:10.986 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:28:32.446 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:11.001 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:28:32.461 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:11.042 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:28:32.525 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:11.046 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:28:32.530 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:11.100 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:28:32.559 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:11.105 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:28:32.565 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:11.133 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:28:32.616 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:58:57.565 INFO:teuthology.orchestra.run.smithi033.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... ok 2019-12-09T13:58:59.231 INFO:teuthology.orchestra.run.smithi033.stderr:testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ok 2019-12-09T13:59:25.493 INFO:teuthology.orchestra.run.smithi033.stderr:testAccountHead (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T13:59:37.368 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerListing (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T13:59:41.445 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:31:02.905 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.450 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:31:02.910 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.458 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:31:02.941 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.459 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:31:02.919 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.462 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:31:02.923 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.470 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:31:02.953 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.574 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:31:03.033 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.584 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:31:03.044 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T13:59:41.597 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:31:03.080 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:14.400 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerSerializedInfo (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:03:26.654 INFO:teuthology.orchestra.run.smithi033.stderr:testContainersOrderedByName (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:03:27.843 INFO:teuthology.orchestra.run.smithi033.stderr:testInvalidAuthToken (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:03:41.301 INFO:teuthology.orchestra.run.smithi033.stderr:testLastContainerMarker (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:03:55.499 INFO:teuthology.orchestra.run.smithi033.stderr:testListingLimit (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:04:52.348 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:36:13.808 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.353 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:36:13.813 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.366 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:36:13.826 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.371 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:36:13.831 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.378 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:36:13.861 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.384 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:36:13.867 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.596 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:36:14.079 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.599 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:36:14.059 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:04:52.621 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:36:14.080 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:03.081 INFO:teuthology.orchestra.run.smithi033.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:05:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:testPUT (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T14:05:32.429 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:36:53.889 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.487 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:36:53.947 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.522 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:36:53.981 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.529 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:36:53.989 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.536 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:36:53.996 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.573 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:36:54.056 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.577 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:36:54.061 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.580 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:36:54.040 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:05:32.644 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:36:54.127 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.329 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:41:25.788 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.334 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:41:25.794 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.344 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:41:25.804 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.349 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:41:25.809 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.359 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:41:25.842 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.364 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:41:25.847 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.520 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:41:25.980 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.535 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:41:25.995 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:04.552 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:41:26.035 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:10:43.568 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainer) ... ok 2019-12-09T14:10:45.116 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T14:10:45.925 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:10:47.300 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T14:10:47.466 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerInfo (test.functional.tests.TestContainer) ... ok 2019-12-09T14:10:47.610 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:10:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerNameLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T14:11:00.264 INFO:teuthology.orchestra.run.smithi033.stderr:testCreate (test.functional.tests.TestContainer) ... ok 2019-12-09T14:11:09.405 INFO:teuthology.orchestra.run.smithi033.stderr:testDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T14:11:10.091 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:11:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainer) ... ok 2019-12-09T14:11:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T14:12:03.641 INFO:teuthology.orchestra.run.smithi033.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainer) ... ok 2019-12-09T14:12:05.068 INFO:teuthology.orchestra.run.smithi033.stderr:testFileOrder (test.functional.tests.TestContainer) ... ok 2019-12-09T14:12:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:testFileThenContainerDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T14:12:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:testLastFileMarker (test.functional.tests.TestContainer) ... ok 2019-12-09T14:12:42.160 INFO:teuthology.orchestra.run.smithi033.stderr:testMarkerLimitFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T14:13:27.256 INFO:teuthology.orchestra.run.smithi033.stderr:testPrefixAndLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T14:13:27.336 INFO:teuthology.orchestra.run.smithi033.stderr:testTooLongName (test.functional.tests.TestContainer) ... ok 2019-12-09T14:13:53.972 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:45:15.432 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:53.978 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:45:15.438 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.035 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:45:15.495 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.042 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:45:15.501 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.045 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:45:15.528 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.047 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:45:15.530 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.107 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:45:15.567 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.149 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:45:15.632 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:54.163 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:45:15.623 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:14:11.162 INFO:teuthology.orchestra.run.smithi033.stderr:testUnorderedList (test.functional.tests.TestContainer) ... ok 2019-12-09T14:15:15.308 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:46:36.767 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.313 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:46:36.773 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.326 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:46:36.786 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.328 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:46:36.811 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.334 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:46:36.793 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.334 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:46:36.817 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.384 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:46:36.844 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.405 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:46:36.865 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:15:15.409 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:46:36.892 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerListing (test.functional.tests.TestContainerPaths) ... ok 2019-12-09T14:17:55.715 INFO:teuthology.orchestra.run.smithi033.stderr:testStructure (test.functional.tests.TestContainerPaths) ... ok 2019-12-09T14:18:07.738 INFO:teuthology.orchestra.run.smithi033.stderr:testTraverseContainer (test.functional.tests.TestContainerPaths) ... ok 2019-12-09T14:19:25.773 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:28.628 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerFileList (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:29.941 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:31.723 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:32.072 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerInfo (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:32.661 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:41.422 INFO:teuthology.orchestra.run.smithi033.stderr:testContainerNameLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:46.111 INFO:teuthology.orchestra.run.smithi033.stderr:testCreate (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:53.669 INFO:teuthology.orchestra.run.smithi033.stderr:testDelete (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:53.781 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:19:59.860 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:20:00.736 INFO:teuthology.orchestra.run.smithi033.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:20:27.901 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:51:49.360 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:27.905 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:51:49.366 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:27.917 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:51:49.377 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:27.919 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:51:49.379 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:27.924 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:51:49.407 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:27.939 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:51:49.422 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:28.099 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:51:49.559 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:28.176 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:51:49.636 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:28.178 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:51:49.661 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:59.785 INFO:teuthology.orchestra.run.smithi033.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:21:01.186 INFO:teuthology.orchestra.run.smithi033.stderr:testFileOrder (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:21:11.771 INFO:teuthology.orchestra.run.smithi033.stderr:testFileThenContainerDelete (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:21:15.970 INFO:teuthology.orchestra.run.smithi033.stderr:testLastFileMarker (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:21:39.083 INFO:teuthology.orchestra.run.smithi033.stderr:testMarkerLimitFileList (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:22:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:testPrefixAndLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:22:15.000 INFO:teuthology.orchestra.run.smithi033.stderr:testTooLongName (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:22:15.415 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:53:36.875 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.421 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:53:36.881 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.464 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:53:36.924 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.466 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:53:36.926 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.479 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:53:36.962 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.483 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:53:36.965 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.497 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:53:36.956 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.520 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:53:36.980 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:22:15.563 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:53:37.046 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:08.017 INFO:teuthology.orchestra.run.smithi033.stderr:testUnorderedList (test.functional.tests.TestContainerUTF8) ... ok 2019-12-09T14:25:40.111 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:57:01.571 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.125 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:57:01.578 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.125 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:57:01.584 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.126 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:57:01.609 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.130 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:57:01.590 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.131 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:57:01.614 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.261 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:04:57:01.721 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.285 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:04:57:01.743 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:25:40.290 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:04:57:01.773 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:26.518 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_manifest (test.functional.tests.TestDlo) ... ok 2019-12-09T14:26:26.777 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_manifest_document_itself (test.functional.tests.TestDlo) ... ok 2019-12-09T14:26:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_range (test.functional.tests.TestDlo) ... ok 2019-12-09T14:26:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_range_out_of_range (test.functional.tests.TestDlo) ... ok 2019-12-09T14:27:16.932 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_manifest (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:27:17.214 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_manifest_document_itself (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:27:21.968 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_range (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:27:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:test_get_range_out_of_range (test.functional.tests.TestDloUTF8) ... ok 2019-12-09T14:27:48.537 INFO:teuthology.orchestra.run.smithi033.stderr:testBadHeaders (test.functional.tests.TestFile) ... ok 2019-12-09T14:27:48.647 INFO:teuthology.orchestra.run.smithi033.stderr:testBlankMetadataName (test.functional.tests.TestFile) ... ok 2019-12-09T14:30:37.031 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:01:58.491 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.058 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:01:58.519 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.070 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:01:58.530 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.072 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:01:58.532 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.107 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:01:58.590 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.116 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:01:58.599 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.195 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:01:58.656 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.200 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:01:58.661 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:37.341 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:01:58.824 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.286 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:02:11.746 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.289 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:02:11.749 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.310 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:02:11.793 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.317 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:02:11.800 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.321 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:02:11.781 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.325 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:02:11.785 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.327 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:02:11.787 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.361 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:02:11.821 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:30:50.387 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:02:11.870 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:15.326 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:32:15.318+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.261564+0000 front 2019-12-09T14:31:33.466719+0000 (oldest deadline 2019-12-09T14:32:14.397027+0000) 2019-12-09T14:32:15.473 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:32:15.461+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.378169+0000 front 2019-12-09T14:31:37.386973+0000 (oldest deadline 2019-12-09T14:32:14.450571+0000) 2019-12-09T14:32:15.521 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:32:15.509+0000 1694f700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:33.477772+0000 front 2019-12-09T14:31:33.508323+0000 (oldest deadline 2019-12-09T14:32:14.572093+0000) 2019-12-09T14:32:16.361 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:32:16.356+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.261564+0000 front 2019-12-09T14:31:33.466719+0000 (oldest deadline 2019-12-09T14:32:14.397027+0000) 2019-12-09T14:32:16.487 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:32:16.475+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.378169+0000 front 2019-12-09T14:31:37.386973+0000 (oldest deadline 2019-12-09T14:32:14.450571+0000) 2019-12-09T14:32:16.499 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:32:16.495+0000 1694f700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:33.477772+0000 front 2019-12-09T14:31:33.508323+0000 (oldest deadline 2019-12-09T14:32:14.572093+0000) 2019-12-09T14:32:17.361 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:32:17.356+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.261564+0000 front 2019-12-09T14:31:33.466719+0000 (oldest deadline 2019-12-09T14:32:14.397027+0000) 2019-12-09T14:32:17.531 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:32:17.529+0000 1694f700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:33.477772+0000 front 2019-12-09T14:31:33.508323+0000 (oldest deadline 2019-12-09T14:32:14.572093+0000) 2019-12-09T14:32:17.540 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:32:17.534+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.378169+0000 front 2019-12-09T14:31:37.386973+0000 (oldest deadline 2019-12-09T14:32:14.450571+0000) 2019-12-09T14:32:18.342 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:32:18.337+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.261564+0000 front 2019-12-09T14:31:33.466719+0000 (oldest deadline 2019-12-09T14:32:14.397027+0000) 2019-12-09T14:32:18.520 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:32:18.516+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.378169+0000 front 2019-12-09T14:31:37.386973+0000 (oldest deadline 2019-12-09T14:32:14.450571+0000) 2019-12-09T14:32:18.588 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:32:18.578+0000 1694f700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:33.477772+0000 front 2019-12-09T14:31:33.508323+0000 (oldest deadline 2019-12-09T14:32:14.572093+0000) 2019-12-09T14:32:19.042 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:32:19.030+0000 1694f700 -1 osd.1 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:33.504045+0000 front 2019-12-09T14:31:37.783426+0000 (oldest deadline 2019-12-09T14:32:18.692155+0000) 2019-12-09T14:32:19.353 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:32:19.344+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.261564+0000 front 2019-12-09T14:31:33.466719+0000 (oldest deadline 2019-12-09T14:32:14.397027+0000) 2019-12-09T14:32:19.530 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:32:19.526+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:37.378169+0000 front 2019-12-09T14:31:37.386973+0000 (oldest deadline 2019-12-09T14:32:14.450571+0000) 2019-12-09T14:32:19.609 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:32:19.604+0000 1694f700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:31:33.477772+0000 front 2019-12-09T14:31:33.508323+0000 (oldest deadline 2019-12-09T14:32:14.572093+0000) 2019-12-09T14:32:57.798 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:04:19.258 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:57.842 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:04:19.302 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:57.843 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:04:19.326 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:57.850 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:04:19.310 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:57.859 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:04:19.342 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:57.871 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:04:19.331 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:57.991 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:04:19.474 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:57.995 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:04:19.455 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:32:58.010 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:04:19.470 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:33:16.993 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:33:16.987+0000 1694f700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.470262+0000 front 2019-12-09T14:32:33.643819+0000 (oldest deadline 2019-12-09T14:33:16.964924+0000) 2019-12-09T14:33:17.932 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:33:17.927+0000 1694f700 -1 osd.4 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:32.194176+0000 front 2019-12-09T14:32:32.249517+0000 (oldest deadline 2019-12-09T14:33:17.797801+0000) 2019-12-09T14:33:17.979 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:33:17.972+0000 1694f700 -1 osd.1 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.615311+0000 front 2019-12-09T14:32:33.644797+0000 (oldest deadline 2019-12-09T14:33:17.954121+0000) 2019-12-09T14:33:18.027 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:33:18.021+0000 1694f700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.470262+0000 front 2019-12-09T14:32:33.643819+0000 (oldest deadline 2019-12-09T14:33:16.964924+0000) 2019-12-09T14:33:18.522 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:33:18.510+0000 1694f700 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.661634+0000 front 2019-12-09T14:32:33.665796+0000 (oldest deadline 2019-12-09T14:33:18.350734+0000) 2019-12-09T14:33:18.812 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:33:18.802+0000 1694f700 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:36.297542+0000 front 2019-12-09T14:32:36.216749+0000 (oldest deadline 2019-12-09T14:33:17.756220+0000) 2019-12-09T14:33:18.947 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:33:18.942+0000 1694f700 -1 osd.4 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:32.194176+0000 front 2019-12-09T14:32:32.249517+0000 (oldest deadline 2019-12-09T14:33:17.797801+0000) 2019-12-09T14:33:18.987 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:33:18.983+0000 1694f700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.470262+0000 front 2019-12-09T14:32:33.643819+0000 (oldest deadline 2019-12-09T14:33:16.964924+0000) 2019-12-09T14:33:19.019 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:33:19.015+0000 1694f700 -1 osd.1 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.615311+0000 front 2019-12-09T14:32:33.644797+0000 (oldest deadline 2019-12-09T14:33:17.954121+0000) 2019-12-09T14:33:19.557 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:33:19.542+0000 1694f700 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.661634+0000 front 2019-12-09T14:32:33.665796+0000 (oldest deadline 2019-12-09T14:33:18.350734+0000) 2019-12-09T14:33:19.786 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:33:19.781+0000 1694f700 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:36.297542+0000 front 2019-12-09T14:32:36.216749+0000 (oldest deadline 2019-12-09T14:33:17.756220+0000) 2019-12-09T14:33:19.909 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:33:19.903+0000 1694f700 -1 osd.4 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:32.194176+0000 front 2019-12-09T14:32:32.249517+0000 (oldest deadline 2019-12-09T14:33:17.797801+0000) 2019-12-09T14:33:19.990 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:33:19.984+0000 1694f700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.470262+0000 front 2019-12-09T14:32:33.643819+0000 (oldest deadline 2019-12-09T14:33:16.964924+0000) 2019-12-09T14:33:20.003 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:33:19.987+0000 1694f700 -1 osd.1 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.615311+0000 front 2019-12-09T14:32:33.644797+0000 (oldest deadline 2019-12-09T14:33:17.954121+0000) 2019-12-09T14:33:20.595 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:33:20.588+0000 1694f700 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:32:33.661634+0000 front 2019-12-09T14:32:33.665796+0000 (oldest deadline 2019-12-09T14:33:18.350734+0000) 2019-12-09T14:33:50.928 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:33:50.922+0000 1694f700 -1 osd.4 147 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102265 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 821426992 gen 56] snapc 0=[] ondisk+write+known_if_redirected e145) 2019-12-09T14:35:04.425 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:06:25.885 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.431 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:06:25.892 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.434 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:06:25.895 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.443 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:06:25.926 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.448 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:06:25.908 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.450 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:06:25.934 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.610 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:06:26.065 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.610 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:06:26.065 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:04.610 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:06:26.091 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:35:32.501 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:35:32.499+0000 1694f700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:49.740736+0000 front 2019-12-09T14:34:49.666651+0000 (oldest deadline 2019-12-09T14:35:32.298323+0000) 2019-12-09T14:35:32.992 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:35:32.979+0000 1694f700 -1 osd.2 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.309654+0000 front 2019-12-09T14:34:48.256715+0000 (oldest deadline 2019-12-09T14:35:32.791558+0000) 2019-12-09T14:35:33.441 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:35:33.436+0000 1694f700 -1 osd.4 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.782747+0000 front 2019-12-09T14:34:48.779509+0000 (oldest deadline 2019-12-09T14:35:33.409152+0000) 2019-12-09T14:35:33.539 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:35:33.536+0000 1694f700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:49.740736+0000 front 2019-12-09T14:34:49.666651+0000 (oldest deadline 2019-12-09T14:35:32.298323+0000) 2019-12-09T14:35:33.990 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:35:33.984+0000 1694f700 -1 osd.2 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.309654+0000 front 2019-12-09T14:34:48.256715+0000 (oldest deadline 2019-12-09T14:35:32.791558+0000) 2019-12-09T14:35:34.239 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:35:34.236+0000 1694f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:52.919212+0000 front 2019-12-09T14:34:52.400974+0000 (oldest deadline 2019-12-09T14:35:34.111627+0000) 2019-12-09T14:35:34.321 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:35:34.316+0000 1694f700 -1 osd.0 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.179564+0000 front 2019-12-09T14:34:49.941049+0000 (oldest deadline 2019-12-09T14:35:33.834790+0000) 2019-12-09T14:35:34.379 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:35:34.374+0000 1694f700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.725105+0000 front 2019-12-09T14:34:48.443070+0000 (oldest deadline 2019-12-09T14:35:34.194915+0000) 2019-12-09T14:35:34.414 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:35:34.409+0000 1694f700 -1 osd.4 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.782747+0000 front 2019-12-09T14:34:48.779509+0000 (oldest deadline 2019-12-09T14:35:33.409152+0000) 2019-12-09T14:35:34.581 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:35:34.577+0000 1694f700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:49.740736+0000 front 2019-12-09T14:34:49.666651+0000 (oldest deadline 2019-12-09T14:35:32.298323+0000) 2019-12-09T14:35:34.987 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:35:34.981+0000 1694f700 -1 osd.2 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.309654+0000 front 2019-12-09T14:34:48.256715+0000 (oldest deadline 2019-12-09T14:35:32.791558+0000) 2019-12-09T14:35:35.264 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:35:35.260+0000 1694f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:52.919212+0000 front 2019-12-09T14:34:52.400974+0000 (oldest deadline 2019-12-09T14:35:34.111627+0000) 2019-12-09T14:35:35.366 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:35:35.361+0000 1694f700 -1 osd.0 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.179564+0000 front 2019-12-09T14:34:49.941049+0000 (oldest deadline 2019-12-09T14:35:33.834790+0000) 2019-12-09T14:35:35.368 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:35:35.362+0000 1694f700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.725105+0000 front 2019-12-09T14:34:48.443070+0000 (oldest deadline 2019-12-09T14:35:34.194915+0000) 2019-12-09T14:35:35.376 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:35:35.372+0000 1694f700 -1 osd.4 150 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:34:48.782747+0000 front 2019-12-09T14:34:48.779509+0000 (oldest deadline 2019-12-09T14:35:33.409152+0000) 2019-12-09T14:36:07.355 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:07.352+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:08.321 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:08.318+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:09.330 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:09.328+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:10.342 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:10.339+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:11.382 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:11.380+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:12.389 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:12.379+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:13.374 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:13.372+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:14.341 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:14.338+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:15.393 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:15.390+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:16.393 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:16.390+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:17.368 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:17.364+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:18.322 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:18.319+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:19.341 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:19.339+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:20.322 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:20.319+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:21.346 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:21.342+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:22.364 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:22.361+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:23.413 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:23.410+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:24.378 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:24.375+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:25.349 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:25.346+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:26.310 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:26.306+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:27.335 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:27.332+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:28.311 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:28.306+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:29.269 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:29.266+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:30.304 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:30.299+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:31.298 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:31.294+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:32.266 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:32.263+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:33.242 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:33.239+0000 1694f700 -1 osd.4 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:46.804230+0000 front 2019-12-09T14:35:46.835349+0000 (oldest deadline 2019-12-09T14:36:32.687234+0000) 2019-12-09T14:36:33.243 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:33.240+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:33.367 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:36:33.362+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:49.243645+0000 front 2019-12-09T14:35:49.183513+0000 (oldest deadline 2019-12-09T14:36:32.680409+0000) 2019-12-09T14:36:34.003 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:36:33.998+0000 1694f700 -1 osd.7 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:48.828075+0000 front 2019-12-09T14:35:48.987613+0000 (oldest deadline 2019-12-09T14:36:33.481769+0000) 2019-12-09T14:36:34.212 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:34.209+0000 1694f700 -1 osd.4 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:46.804230+0000 front 2019-12-09T14:35:46.835349+0000 (oldest deadline 2019-12-09T14:36:32.687234+0000) 2019-12-09T14:36:34.214 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:34.211+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:34.327 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:36:34.321+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:49.243645+0000 front 2019-12-09T14:35:49.183513+0000 (oldest deadline 2019-12-09T14:36:32.680409+0000) 2019-12-09T14:36:35.039 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:36:35.034+0000 1694f700 -1 osd.7 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:48.828075+0000 front 2019-12-09T14:35:48.987613+0000 (oldest deadline 2019-12-09T14:36:33.481769+0000) 2019-12-09T14:36:35.176 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:35.168+0000 1694f700 -1 osd.4 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:46.804230+0000 front 2019-12-09T14:35:46.835349+0000 (oldest deadline 2019-12-09T14:36:32.687234+0000) 2019-12-09T14:36:35.177 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:35.170+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:35.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:36:35.320+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:49.243645+0000 front 2019-12-09T14:35:49.183513+0000 (oldest deadline 2019-12-09T14:36:32.680409+0000) 2019-12-09T14:36:35.454 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:36:35.449+0000 1694f700 -1 osd.1 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:51.065021+0000 front 2019-12-09T14:35:50.744802+0000 (oldest deadline 2019-12-09T14:36:34.814687+0000) 2019-12-09T14:36:36.037 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:36:36.035+0000 1694f700 -1 osd.7 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:48.828075+0000 front 2019-12-09T14:35:48.987613+0000 (oldest deadline 2019-12-09T14:36:33.481769+0000) 2019-12-09T14:36:36.131 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:36.128+0000 1694f700 -1 osd.4 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:46.804230+0000 front 2019-12-09T14:35:46.835349+0000 (oldest deadline 2019-12-09T14:36:32.687234+0000) 2019-12-09T14:36:36.131 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:36.130+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:36.343 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:36:36.338+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:49.243645+0000 front 2019-12-09T14:35:49.183513+0000 (oldest deadline 2019-12-09T14:36:32.680409+0000) 2019-12-09T14:36:36.429 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:36:36.421+0000 1694f700 -1 osd.1 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:51.065021+0000 front 2019-12-09T14:35:50.744802+0000 (oldest deadline 2019-12-09T14:36:34.814687+0000) 2019-12-09T14:36:36.992 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:36:36.984+0000 1694f700 -1 osd.3 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:51.287971+0000 front 2019-12-09T14:35:51.419547+0000 (oldest deadline 2019-12-09T14:36:35.953423+0000) 2019-12-09T14:36:36.995 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:36:36.991+0000 1694f700 -1 osd.7 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:48.828075+0000 front 2019-12-09T14:35:48.987613+0000 (oldest deadline 2019-12-09T14:36:33.481769+0000) 2019-12-09T14:36:37.132 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:37.126+0000 1694f700 -1 osd.4 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:46.804230+0000 front 2019-12-09T14:35:46.835349+0000 (oldest deadline 2019-12-09T14:36:32.687234+0000) 2019-12-09T14:36:37.133 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:37.128+0000 1694f700 -1 osd.4 152 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:37.399 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:36:37.390+0000 1694f700 -1 osd.0 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:49.243645+0000 front 2019-12-09T14:35:49.183513+0000 (oldest deadline 2019-12-09T14:36:32.680409+0000) 2019-12-09T14:36:37.402 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:36:37.395+0000 1694f700 -1 osd.1 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:51.065021+0000 front 2019-12-09T14:35:50.744802+0000 (oldest deadline 2019-12-09T14:36:34.814687+0000) 2019-12-09T14:36:37.946 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:36:37.942+0000 1694f700 -1 osd.3 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:51.287971+0000 front 2019-12-09T14:35:51.419547+0000 (oldest deadline 2019-12-09T14:36:35.953423+0000) 2019-12-09T14:36:37.973 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:36:37.970+0000 1694f700 -1 osd.7 152 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:35:48.828075+0000 front 2019-12-09T14:35:48.987613+0000 (oldest deadline 2019-12-09T14:36:33.481769+0000) 2019-12-09T14:36:38.243 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:38.178+0000 1694f700 -1 osd.4 153 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:39.424 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:36:39.357+0000 1694f700 -1 osd.4 154 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102499 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e151) 2019-12-09T14:36:55.244 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:36:55.233+0000 32788700 -1 osd.5 157 build_incremental_map_msg missing incremental map 157 2019-12-09T14:36:55.301 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:36:55.296+0000 32f89700 -1 osd.5 157 build_incremental_map_msg missing incremental map 157 2019-12-09T14:37:48.931 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:37:48.783+0000 36f91700 -1 osd.2 160 build_incremental_map_msg missing incremental map 160 2019-12-09T14:37:48.971 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:37:48.803+0000 36f91700 -1 osd.2 160 build_incremental_map_msg missing incremental map 160 2019-12-09T14:37:48.977 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:37:48.815+0000 36f91700 -1 osd.2 160 build_incremental_map_msg missing incremental map 160 2019-12-09T14:37:49.013 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:37:48.818+0000 36f91700 -1 osd.2 160 build_incremental_map_msg missing incremental map 160 2019-12-09T14:37:49.031 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:37:48.826+0000 36f91700 -1 osd.2 160 build_incremental_map_msg missing incremental map 160 2019-12-09T14:37:53.151 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:37:53.136+0000 32f89700 -1 osd.5 161 build_incremental_map_msg missing incremental map 157 2019-12-09T14:37:57.731 INFO:teuthology.orchestra.run.smithi033.stderr:testChunkedPut (test.functional.tests.TestFile) ... ok 2019-12-09T14:38:04.551 INFO:teuthology.orchestra.run.smithi033.stderr:testContentTypeGuessing (test.functional.tests.TestFile) ... ok 2019-12-09T14:38:19.157 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:38:19.035+0000 1694f700 -1 osd.5 161 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e159) 2019-12-09T14:38:20.537 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:38:20.276+0000 1694f700 -1 osd.5 161 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e159) 2019-12-09T14:38:21.483 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:38:21.301+0000 1694f700 -1 osd.5 161 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e159) 2019-12-09T14:38:48.680 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:48.677+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:49.682 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:49.678+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:50.665 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:50.662+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:51.674 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:51.671+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:52.673 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:52.670+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:53.695 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:53.692+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:54.715 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:54.712+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:55.717 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:55.715+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:56.694 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:56.690+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:57.738 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:57.735+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:58.712 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:58.709+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:38:59.048 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:10:20.508 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.056 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:10:20.517 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.108 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:10:20.568 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.110 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:10:20.570 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.126 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:10:20.609 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.129 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:10:20.612 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.305 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:10:20.765 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.338 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:10:20.798 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.381 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:10:20.861 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:38:59.681 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:38:59.678+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:00.718 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:00.716+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:01.748 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:01.745+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:02.749 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:02.747+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:03.705 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:03.702+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:04.694 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:04.688+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:05.694 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:05.691+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:06.738 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:06.734+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:07.718 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:07.715+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:08.598 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:10:30.058 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.600 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:10:30.060 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.616 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:10:30.099 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.621 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:10:30.082 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.626 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:10:30.086 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.634 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:10:30.116 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.647 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:10:30.102 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.675 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:10:30.135 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.679 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:10:30.162 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:08.768 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:08.766+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:09.722 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:09.718+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:10.697 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:10.693+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:11.689 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:11.686+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:12.464 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:12.457+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:12.714 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:12.710+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:12.716 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:12.713+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:13.485 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:13.480+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:13.714 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:13.709+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:13.714 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:13.711+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:13.872 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:13.866+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:13.942 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:13.939+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:14.073 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:14.066+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:14.513 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:14.508+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:14.681 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:14.677+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:14.682 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:14.678+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:14.865 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:14.860+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:14.916 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:14.912+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:15.059 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:15.054+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:15.553 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:15.548+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:15.706 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:15.701+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:15.708 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:15.703+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:15.893 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:15.889+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:15.924 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:15.919+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:16.029 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:16.014+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:16.278 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:16.273+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:16.513 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:16.508+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:16.743 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:16.738+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:16.743 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:16.740+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:16.908 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:16.904+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:16.920 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:16.915+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:16.991 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:16.985+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:17.277 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:17.272+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:17.540 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:17.536+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:17.701 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:17.696+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:17.702 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:17.698+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:17.883 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:17.877+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:17.957 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:17.954+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:18.014 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:18.008+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:18.290 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:18.283+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:18.556 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:18.552+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:18.733 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:18.729+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:18.733 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:18.731+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:18.906 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:18.900+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:18.980 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:18.977+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:19.031 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:19.025+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:19.330 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:19.325+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:19.546 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:19.542+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:19.696 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:19.690+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:19.697 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:19.695+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:19.896 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:19.890+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:19.945 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:19.942+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:19.985 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:19.980+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:20.333 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:20.328+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:20.559 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:20.555+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:20.725 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:20.722+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:20.727 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:20.725+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:20.906 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:20.900+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:20.955 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:20.952+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:21.006 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:21.002+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:21.368 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:21.363+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:21.517 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:21.512+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:21.699 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:21.695+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:21.699 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:21.697+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:21.939 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:21.936+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:21.942 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:21.937+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:22.016 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:22.010+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:22.391 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:22.386+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:22.549 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:22.544+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:22.735 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:22.728+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:22.735 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:22.728+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:22.908 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:22.901+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:22.970 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:22.966+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:22.972 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:22.967+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:23.352 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:23.346+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:23.520 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:23.513+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:23.739 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:23.733+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:23.740 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:23.737+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:23.874 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:23.868+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:23.964 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:23.959+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:23.994 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:23.990+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:24.306 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:24.301+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:24.519 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:24.515+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:24.784 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:24.780+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:24.788 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:24.781+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:24.846 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:24.842+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:25.011 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:25.006+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:25.022 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:25.019+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:25.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:25.299+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:25.474 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:25.470+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:25.776 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:25.772+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:25.778 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:25.775+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:25.883 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:25.878+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:25.980 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:25.976+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:25.981 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:25.975+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:26.302 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:26.296+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:26.487 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:26.481+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:26.775 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:26.770+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:26.775 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:26.772+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:26.903 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:26.898+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:26.987 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:26.984+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:26.993 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:26.989+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:27.301 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:27.295+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:27.483 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:27.478+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:27.792 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:27.787+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:27.792 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:27.789+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:27.935 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:27.930+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:27.999 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:27.987+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:28.044 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:28.039+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:28.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:28.320+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:28.490 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:28.485+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:28.822 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:28.817+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:28.822 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:28.819+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:28.987 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:28.983+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:28.988 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:28.982+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:29.002 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:28.996+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:29.296 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:29.292+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:29.478 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:29.472+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:29.876 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:29.872+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:29.877 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:29.873+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:29.956 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:29.950+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:29.986 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:29.982+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:30.018 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:30.013+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:30.306 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:30.300+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:30.488 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:30.482+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:30.851 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:30.847+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:30.853 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:30.849+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:30.937 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:30.930+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:31.002 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:30.999+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:31.022 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:31.017+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:31.331 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:31.326+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:31.499 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:31.494+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:31.810 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:31.807+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:31.814 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:31.810+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:31.950 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:31.943+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:31.999 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:31.995+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:32.039 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:32.034+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:32.340 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:32.335+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:32.466 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:32.462+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:32.849 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:32.844+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:32.849 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:32.846+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:32.912 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:32.907+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:33.023 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:33.018+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:33.031 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:33.027+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:33.323 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:33.318+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:33.443 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:33.439+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:33.897 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:33.893+0000 1694f700 -1 osd.4 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.409794+0000 front 2019-12-09T14:38:30.088642+0000 (oldest deadline 2019-12-09T14:39:12.080957+0000) 2019-12-09T14:39:33.898 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:33.894+0000 1694f700 -1 osd.4 161 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:33.961 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:39:33.954+0000 1694f700 -1 osd.3 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.550881+0000 front 2019-12-09T14:38:29.552086+0000 (oldest deadline 2019-12-09T14:39:13.748918+0000) 2019-12-09T14:39:33.982 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:39:33.976+0000 1694f700 -1 osd.2 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:30.551191+0000 front 2019-12-09T14:38:29.549096+0000 (oldest deadline 2019-12-09T14:39:13.723091+0000) 2019-12-09T14:39:34.073 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:39:34.069+0000 1694f700 -1 osd.7 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.171727+0000 front 2019-12-09T14:38:29.063947+0000 (oldest deadline 2019-12-09T14:39:13.793277+0000) 2019-12-09T14:39:34.337 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:39:34.333+0000 1694f700 -1 osd.0 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:31.700915+0000 front 2019-12-09T14:38:32.105945+0000 (oldest deadline 2019-12-09T14:39:16.095358+0000) 2019-12-09T14:39:34.423 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:39:34.417+0000 1694f700 -1 osd.1 161 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:38:29.059489+0000 front 2019-12-09T14:38:30.611070+0000 (oldest deadline 2019-12-09T14:39:11.763671+0000) 2019-12-09T14:39:35.161 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:35.159+0000 1694f700 -1 osd.4 162 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:39:36.235 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:39:36.228+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102835 2.23s0 2:c5d4c61f:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_2cac92bb72614de998589af671fe4bbc:head [create,setxattr user.rgw.idtag (33) in=47b,setxattr user.rgw.tail_tag (33) in=50b,writefull 0~17287 in=17287b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-b60c072b15c74eb3ad246bfadc00b270 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T14:40:07.075 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:07.071+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:08.109 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:08.104+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:09.143 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:09.139+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:10.115 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:10.113+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:11.138 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:11.136+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:12.129 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:12.127+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:13.105 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:13.103+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:14.147 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:14.145+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:15.135 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:15.133+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:16.100 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:16.097+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:17.099 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:17.096+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:18.127 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:18.123+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:19.113 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:19.110+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:20.072 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:20.069+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:21.038 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:21.034+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:22.030 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:22.027+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:23.059 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:23.056+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:24.110 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:24.107+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:25.152 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:25.148+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:26.153 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:26.150+0000 1694f700 -1 osd.4 163 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:27.125 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:27.122+0000 1694f700 -1 osd.4 164 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:28.109 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:28.106+0000 1694f700 -1 osd.4 164 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:29.340 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:29.337+0000 1694f700 -1 osd.4 165 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:30.355 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:30.348+0000 1694f700 -1 osd.4 166 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:30.876 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:40:30.858+0000 31f87700 -1 osd.5 166 build_incremental_map_msg missing incremental map 166 2019-12-09T14:40:31.487 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:31.393+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:32.366 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:32.363+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:32.739 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:32.733+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:33.309 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:33.305+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:33.351 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:33.347+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:33.351 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:33.348+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:33.537 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:33.532+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:33.739 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:33.733+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:33.848 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:33.843+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:34.340 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:34.335+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:34.353 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:34.348+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:34.353 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:34.350+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:34.503 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:34.498+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:34.719 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:34.713+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:34.911 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:34.892+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:35.337 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:35.334+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:35.338 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:35.336+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:35.339 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:35.332+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:35.502 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:35.498+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:35.713 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:35.708+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:35.916 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:35.912+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:36.253 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:36.248+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:36.389 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:36.384+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:36.389 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:36.386+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:36.419 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:36.414+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:36.463 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:36.459+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:36.721 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:36.716+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:36.914 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:36.908+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:37.257 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:37.252+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:37.397 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:37.391+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:37.430 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:37.426+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:37.433 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:37.430+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:37.483 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:37.478+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:37.720 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:37.715+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:37.903 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:37.898+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:38.308 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:38.303+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:38.379 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:38.374+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:38.420 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:38.415+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:38.421 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:38.417+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:38.478 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:38.473+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:38.725 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:38.719+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:38.889 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:38.883+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:39.361 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:39.356+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:39.427 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:39.420+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:39.453 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:39.447+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:39.453 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:39.450+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:39.501 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:39.496+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:39.759 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:39.754+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:39.844 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:39.838+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:40.353 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:40.348+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:40.415 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:40.410+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:40.457 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:40.453+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:40.472 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:40.467+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:40.473 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:40.470+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:40.772 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:40.768+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:40.885 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:40.878+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:41.385 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:41.381+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:41.423 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:41.418+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:41.458 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:41.453+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:41.523 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:41.517+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:41.523 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:41.520+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:41.749 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:41.743+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:41.839 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:41.835+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:42.358 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:42.351+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:42.408 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:42.403+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:42.458 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:42.453+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:42.525 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:42.520+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:42.526 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:42.522+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:42.777 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:42.772+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:42.835 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:42.830+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:43.331 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:43.326+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:43.453 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:43.448+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:43.456 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:43.450+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:43.524 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:43.520+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:43.527 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:43.524+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:43.743 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:43.738+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:43.876 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:43.871+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:44.369 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:44.364+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:44.464 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:44.459+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:44.497 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:44.492+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:44.514 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:44.508+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:44.515 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:44.510+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:44.759 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:44.754+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:44.860 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:44.856+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:45.357 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:45.352+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:45.424 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:45.418+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:45.480 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:45.474+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:45.481 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:45.477+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:45.500 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:45.492+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:45.811 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:45.806+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:45.892 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:45.886+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:46.380 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:46.375+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:46.465 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:46.460+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:46.530 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:46.525+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:46.531 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:46.528+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:46.535 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:46.531+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:46.772 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:46.766+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:46.871 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:46.865+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:47.436 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:47.432+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:47.472 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:47.465+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:47.540 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:47.535+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:47.543 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:47.540+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:47.544 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:47.537+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:47.821 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:47.815+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:47.864 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:47.859+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:48.398 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:48.393+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:48.476 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:48.470+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:48.515 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:48.511+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:48.518 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:48.515+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:48.524 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:48.517+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:48.780 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:48.774+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:48.885 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:48.881+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:49.434 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:40:49.430+0000 1694f700 -1 osd.7 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.563993+0000 front 2019-12-09T14:39:50.806754+0000 (oldest deadline 2019-12-09T14:40:35.844684+0000) 2019-12-09T14:40:49.437 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:40:49.432+0000 1694f700 -1 osd.0 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:50.206956+0000 front 2019-12-09T14:39:49.710961+0000 (oldest deadline 2019-12-09T14:40:33.151735+0000) 2019-12-09T14:40:49.501 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:49.497+0000 1694f700 -1 osd.4 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.207340+0000 front 2019-12-09T14:39:49.195363+0000 (oldest deadline 2019-12-09T14:40:33.210518+0000) 2019-12-09T14:40:49.504 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:49.500+0000 1694f700 -1 osd.4 167 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:49.517 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:40:49.511+0000 1694f700 -1 osd.3 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.205156+0000 front 2019-12-09T14:39:49.385853+0000 (oldest deadline 2019-12-09T14:40:32.652795+0000) 2019-12-09T14:40:49.785 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:40:49.778+0000 1694f700 -1 osd.1 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:47.994450+0000 front 2019-12-09T14:39:48.313237+0000 (oldest deadline 2019-12-09T14:40:31.871934+0000) 2019-12-09T14:40:49.921 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:40:49.917+0000 1694f700 -1 osd.6 167 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:39:49.738009+0000 front 2019-12-09T14:39:49.813701+0000 (oldest deadline 2019-12-09T14:40:33.801091+0000) 2019-12-09T14:40:50.101 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:40:49.916+0000 32788700 -1 osd.5 168 build_incremental_map_msg missing incremental map 166 2019-12-09T14:40:50.473 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:50.470+0000 1694f700 -1 osd.4 168 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:40:51.500 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:40:51.449+0000 1694f700 -1 osd.4 169 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:102965 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=326b] snapc 0=[] ondisk+write+known_if_redirected e163) 2019-12-09T14:41:01.507 INFO:teuthology.orchestra.run.smithi033.stderr:testCopy (test.functional.tests.TestFile) ... ok 2019-12-09T14:41:22.826 INFO:teuthology.orchestra.run.smithi033.stderr:testCopy404s (test.functional.tests.TestFile) ... ok 2019-12-09T14:41:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyDestinationSlashProblems (test.functional.tests.TestFile) ... ok 2019-12-09T14:41:41.197 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyFromHeader (test.functional.tests.TestFile) ... ok 2019-12-09T14:41:45.458 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyFromHeader404s (test.functional.tests.TestFile) ... ok 2019-12-09T14:41:46.467 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyNoDestinationHeader (test.functional.tests.TestFile) ... ok 2019-12-09T14:41:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:testDelete (test.functional.tests.TestFile) ... ok 2019-12-09T14:42:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteOfFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T14:42:24.674 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteThen404s (test.functional.tests.TestFile) ... ok 2019-12-09T14:42:25.748 INFO:teuthology.orchestra.run.smithi033.stderr:testEtagResponse (test.functional.tests.TestFile) ... ok 2019-12-09T14:42:26.655 INFO:teuthology.orchestra.run.smithi033.stderr:testEtagWayoff (test.functional.tests.TestFile) ... ok 2019-12-09T14:42:48.458 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:48.454+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:48.898 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:48.893+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:49.509 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:49.505+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:49.875 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:49.870+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:49.964 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:49.958+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:50.496 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:50.492+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:50.688 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:50.680+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:50.920 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:50.917+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:51.020 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:51.012+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:51.540 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:51.536+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:51.712 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:51.706+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:51.884 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:51.881+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:52.013 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:52.007+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:52.423 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:52.416+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:52.502 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:52.497+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:52.684 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:52.677+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:52.845 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:52.840+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:52.864 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:52.860+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:53.009 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:53.001+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:53.420 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:53.414+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:53.467 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:53.464+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:53.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:53.647+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:53.867 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:53.860+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:53.879 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:53.876+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:53.984 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:53.978+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:54.425 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:54.417+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:54.426 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:54.424+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:54.621 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:54.616+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:54.834 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:54.829+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:54.925 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:54.922+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:54.964 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:54.958+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:55.431 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:55.428+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:55.468 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:55.462+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:55.586 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:55.579+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:55.813 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:55.807+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:55.925 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:55.919+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:55.968 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:55.965+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:56.441 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:56.439+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:56.453 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:56.446+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:56.601 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:56.595+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:56.823 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:56.818+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:56.962 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:56.956+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:57.005 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:57.001+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:57.421 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:57.415+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:57.428 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:57.423+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:57.633 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:57.626+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:57.822 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:57.815+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:57.986 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:57.979+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:58.012 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:58.008+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:58.410 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:58.397+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:58.412 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:58.407+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:58.654 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:58.648+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:58.823 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:58.817+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:58.957 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:58.950+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:42:59.031 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:42:59.027+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:42:59.389 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:42:59.380+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:42:59.432 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:42:59.428+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:42:59.613 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:42:59.607+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:42:59.869 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:42:59.862+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:42:59.938 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:42:59.931+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:00.006 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:00.003+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:00.423 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:00.418+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:00.433 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:00.429+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:00.610 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:00.604+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:00.836 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:00.830+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:00.962 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:00.956+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:01.010 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:01.006+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:01.429 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:01.426+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:01.451 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:01.446+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:01.567 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:01.561+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:01.819 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:01.813+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:01.922 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:01.916+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:01.982 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:01.977+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:02.444 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:02.437+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:02.462 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:02.458+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:02.615 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:02.609+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:02.838 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:02.832+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:02.892 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:02.885+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:02.997 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:02.991+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:03.421 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:03.415+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:03.491 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:03.488+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:03.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:03.661+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:03.813 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:03.806+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:03.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:03.872+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:04.025 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:04.020+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:04.426 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:04.420+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:04.462 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:04.458+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:04.688 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:04.683+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:04.843 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:04.838+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:04.929 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:04.923+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:05.069 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:05.066+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:05.142 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:14:26.602 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.153 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:14:26.613 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.155 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:14:26.615 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.159 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:14:26.619 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.160 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:14:26.643 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.164 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:14:26.647 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.404 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:14:26.864 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.411 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:14:26.894 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.450 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:05.445+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:05.494 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:14:26.954 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:43:05.503 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:05.500+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:05.641 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:05.636+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:05.797 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:05.792+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:05.890 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:05.885+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:06.114 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:06.109+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:06.483 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:06.478+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:06.502 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:06.497+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:06.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:06.662+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:06.833 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:06.829+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:06.905 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:06.896+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:07.104 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:07.100+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:07.458 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:07.452+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:07.530 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:07.526+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:07.640 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:07.635+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:07.881 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:07.877+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:07.913 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:07.908+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:08.115 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:08.111+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:08.465 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:08.460+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:08.521 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:08.517+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:08.612 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:08.600+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:08.914 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:08.908+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:08.926 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:08.921+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:09.144 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:09.140+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:09.485 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:09.473+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:09.555 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:09.550+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:09.624 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:09.619+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:09.906 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:09.901+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:09.951 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:09.946+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:10.107 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:10.103+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:10.500 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:10.495+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:10.510 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:10.505+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:10.614 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:10.609+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:10.876 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:10.871+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:11.004 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:10.999+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:11.147 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:11.143+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:11.470 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:11.464+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:11.526 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:11.522+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:11.630 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:11.624+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:11.839 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:11.834+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:11.973 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:11.967+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:12.173 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:12.168+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:12.501 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:12.496+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:12.517 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:12.512+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:12.638 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:12.633+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:12.865 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:12.861+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:12.981 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:12.976+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:13.160 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:13.155+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:13.160 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:13.157+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:13.519 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:13.515+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:13.547 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:13.535+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:13.664 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:13.656+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:13.917 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:13.909+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:13.951 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:13.947+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:14.132 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:14.128+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:14.135 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:14.132+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:14.493 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:14.486+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:14.520 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:14.515+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:14.655 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:14.650+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:14.927 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:14.921+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:14.934 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:14.929+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:15.161 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:15.156+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:15.161 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:15.158+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:15.469 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:15.465+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:15.553 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:15.547+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:15.651 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:15.647+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:15.900 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:15.895+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:15.971 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:15.965+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:16.135 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:16.131+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:16.137 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:16.135+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:16.484 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:16.479+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:16.579 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:16.574+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:16.643 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:16.639+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:16.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:16.876+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:16.973 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:16.969+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:17.097 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:17.093+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:17.098 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:17.095+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:17.484 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:17.481+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:17.609 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:17.604+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:17.659 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:17.654+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:17.909 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:17.904+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:18.008 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:18.003+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:18.106 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:18.101+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:18.106 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:18.103+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:18.483 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:18.479+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:18.651 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:18.647+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:18.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:18.649+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:18.894 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:18.889+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:19.016 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:19.011+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:19.111 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:19.107+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:19.114 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:19.111+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:19.438 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:43:19.434+0000 1694f700 -1 osd.7 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.197937+0000 front 2019-12-09T14:42:08.202123+0000 (oldest deadline 2019-12-09T14:42:47.901349+0000) 2019-12-09T14:43:19.633 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:43:19.627+0000 1694f700 -1 osd.3 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:08.046745+0000 front 2019-12-09T14:42:08.048508+0000 (oldest deadline 2019-12-09T14:42:52.399854+0000) 2019-12-09T14:43:19.652 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:43:19.645+0000 1694f700 -1 osd.1 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.393712+0000 front 2019-12-09T14:42:07.455818+0000 (oldest deadline 2019-12-09T14:42:50.477282+0000) 2019-12-09T14:43:19.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:43:19.845+0000 1694f700 -1 osd.0 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:05.026325+0000 front 2019-12-09T14:42:05.978638+0000 (oldest deadline 2019-12-09T14:42:49.924166+0000) 2019-12-09T14:43:20.039 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:43:20.033+0000 1694f700 -1 osd.2 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:07.768247+0000 front 2019-12-09T14:42:07.515049+0000 (oldest deadline 2019-12-09T14:42:51.982006+0000) 2019-12-09T14:43:20.116 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:43:20.095+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 41sec e172 v172) 2019-12-09T14:43:20.126 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:20.123+0000 1694f700 -1 osd.4 172 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:42:03.009987+0000 front 2019-12-09T14:42:02.937810+0000 (oldest deadline 2019-12-09T14:42:48.201147+0000) 2019-12-09T14:43:20.128 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:20.125+0000 1694f700 -1 osd.4 172 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:21.160 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:21.157+0000 1694f700 -1 osd.4 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:22.135 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:22.132+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103507 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~5927 in=5927b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e172) 2019-12-09T14:43:54.402 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:54.399+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:43:55.385 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:55.381+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:43:56.416 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:56.413+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:43:57.379 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:57.375+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:43:58.403 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:58.399+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:43:59.417 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:43:59.413+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:00.376 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:00.372+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:01.421 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:01.418+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:02.440 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:02.436+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:03.470 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:03.467+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:04.487 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:04.484+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:05.452 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:05.449+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:06.475 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:06.472+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:07.455 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:07.452+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:08.437 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:08.433+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:09.416 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:09.413+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:10.400 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:10.397+0000 1694f700 -1 osd.4 174 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:11.415 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:11.412+0000 1694f700 -1 osd.4 175 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:12.382 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:12.378+0000 1694f700 -1 osd.4 175 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:13.360 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:13.354+0000 1694f700 -1 osd.4 175 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:14.365 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:14.362+0000 1694f700 -1 osd.4 176 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:15.352 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:15.345+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:16.326 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:16.323+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:17.174 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:17.171+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:17.361 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:17.358+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:17.826 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:17.823+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:18.113 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:18.108+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:18.132 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:18.129+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:18.352 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:18.346+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:18.354 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:18.352+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:18.784 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:18.779+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:19.158 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:19.151+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:19.159 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:19.155+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:19.348 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:19.342+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:19.352 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:19.349+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:19.480 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:19.475+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:19.571 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:19.565+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:19.807 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:19.804+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:20.127 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:20.123+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:20.150 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:20.144+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:20.325 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:20.318+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:20.330 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:20.326+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:20.452 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:20.447+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:20.574 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:20.569+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:20.766 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:20.762+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:21.093 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:21.088+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:21.121 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:21.116+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:21.296 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:21.291+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:21.299 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:21.296+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:21.412 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:21.407+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:21.603 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:21.599+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:21.731 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:21.725+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:22.088 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:22.078+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:22.115 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:22.111+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:22.286 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:22.280+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:22.289 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:22.286+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:22.377 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:22.372+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:22.596 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:22.591+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:22.784 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:22.778+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:23.081 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:23.076+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:23.085 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:23.080+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:23.284 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:23.278+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:23.289 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:23.286+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:23.403 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:23.399+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:23.630 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:23.624+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:23.828 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:23.823+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:24.051 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:24.044+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:24.065 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:24.059+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:24.312 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:24.301+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:24.312 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:24.306+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:24.442 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:24.434+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:24.682 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:24.677+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:24.821 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:24.817+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:25.017 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:25.013+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:25.097 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:25.091+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:25.312 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:25.307+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:25.313 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:25.310+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:25.453 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:25.448+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:25.723 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:25.719+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:25.819 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:25.815+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:26.045 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:26.039+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:26.149 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:26.143+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:26.341 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:26.335+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:26.342 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:26.338+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:26.457 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:26.451+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:26.718 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:26.712+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:26.799 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:26.795+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:27.008 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:27.003+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:27.149 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:27.140+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:27.378 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:27.372+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:27.379 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:27.375+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:27.494 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:27.489+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:27.680 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:27.675+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:27.829 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:27.824+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:28.054 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:28.049+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:28.121 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:28.116+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:28.349 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:28.343+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:28.349 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:28.346+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:28.466 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:28.461+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:28.689 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:28.684+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:28.848 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:28.844+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:29.075 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:29.069+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:29.080 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:29.074+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:29.396 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:29.392+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:29.399 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:29.396+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:29.454 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:29.448+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:29.699 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:29.694+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:29.901 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:29.896+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:30.049 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:30.044+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:30.067 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:30.062+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:30.430 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:30.425+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:30.431 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:30.427+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:30.466 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:30.462+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:30.671 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:30.666+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:30.858 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:30.854+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:31.019 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:31.014+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:31.092 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:31.085+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:31.411 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:31.407+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:31.415 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:31.412+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:31.438 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:31.432+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:31.668 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:31.663+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:31.865 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:31.860+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:31.981 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:31.976+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:32.061 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:32.056+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:32.383 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:32.378+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:32.384 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:32.381+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:32.438 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:32.434+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:32.648 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:32.641+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:32.880 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:32.875+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:32.984 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:32.979+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:33.015 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:33.011+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:33.396 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:33.393+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:33.399 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:33.396+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:33.491 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:33.485+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:33.697 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:33.693+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:33.870 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:33.865+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:33.997 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:33.992+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:34.001 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:33.996+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:34.387 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:34.383+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:34.390 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:34.387+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:34.459 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:34.453+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:34.717 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:34.712+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:34.903 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:34.898+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:35.026 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:35.021+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:35.043 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:35.037+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:35.378 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:35.369+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:35.379 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:35.371+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:35.474 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:35.469+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:35.698 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:35.693+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:35.879 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:35.874+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:36.067 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:36.064+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:36.067 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:36.034+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:36.410 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:36.404+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:36.410 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:36.407+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:36.475 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:36.469+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:36.701 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:36.696+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:36.904 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:36.899+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:37.014 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:37.008+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:37.085 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:37.080+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:37.421 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:37.412+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:37.422 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:37.415+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:37.448 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:37.442+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:37.682 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:37.676+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:37.883 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:37.879+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:38.040 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:38.036+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:38.062 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:38.057+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:38.440 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:38.436+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:38.443 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:38.441+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:38.453 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:38.448+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:38.702 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:38.696+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:38.935 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:38.932+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:39.032 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:39.028+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:39.059 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:39.052+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:39.398 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:39.394+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:39.399 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:39.396+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:39.427 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:39.423+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:39.675 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:39.670+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:39.955 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:39.952+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:40.073 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:40.069+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:40.114 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:40.103+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:40.412 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:40.403+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:40.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:40.427+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:40.446 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:40.429+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:40.632 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:40.627+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:40.997 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:40.991+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:41.063 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:41.060+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:41.116 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:41.110+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:41.423 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:41.417+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:41.448 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:41.444+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:41.449 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:41.446+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:41.667 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:41.663+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:41.983 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:41.979+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:42.051 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:42.047+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:42.109 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:42.103+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:42.449 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:42.443+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:42.469 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:42.463+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:42.469 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:42.465+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:42.691 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:42.686+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:42.991 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:42.983+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:43.023 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:43.019+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:43.139 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:43.133+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:43.421 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:43.415+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:43.423 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:43.419+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:43.424 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:43.421+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:43.685 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:43.680+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:43.956 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:43.952+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:43.989 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:43.984+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:44.182 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:44.175+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:44.464 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:44.459+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:44.475 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:44.471+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:44.478 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:44.475+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:44.727 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:44:44.722+0000 1694f700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:35.141141+0000 front 2019-12-09T14:43:34.895514+0000 (oldest deadline 2019-12-09T14:44:18.965210+0000) 2019-12-09T14:44:44.934 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:44:44.929+0000 1694f700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:36.457880+0000 front 2019-12-09T14:43:36.642866+0000 (oldest deadline 2019-12-09T14:44:17.313013+0000) 2019-12-09T14:44:44.963 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:44:44.958+0000 1694f700 -1 osd.6 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:31.977058+0000 front 2019-12-09T14:43:31.867479+0000 (oldest deadline 2019-12-09T14:44:17.117524+0000) 2019-12-09T14:44:45.180 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:44:45.173+0000 1694f700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.315712+0000 front 2019-12-09T14:43:33.403838+0000 (oldest deadline 2019-12-09T14:44:17.323375+0000) 2019-12-09T14:44:45.434 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:44:45.429+0000 1694f700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:33.323827+0000 front 2019-12-09T14:43:33.334494+0000 (oldest deadline 2019-12-09T14:44:18.590278+0000) 2019-12-09T14:44:45.519 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:45.466+0000 1694f700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:43:34.663163+0000 front 2019-12-09T14:43:34.694729+0000 (oldest deadline 2019-12-09T14:44:17.552642+0000) 2019-12-09T14:44:45.525 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:45.469+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:46.153 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:44:46.118+0000 1694f700 -1 osd.5 179 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e176) 2019-12-09T14:44:46.454 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:46.451+0000 1694f700 -1 osd.4 179 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4524.0:103574 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=341b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:44:47.452 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:44:47.439+0000 1694f700 -1 osd.4 180 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:103584 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e174) 2019-12-09T14:45:05.238 INFO:teuthology.orchestra.run.smithi033.stderr:testFileCreate (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:testGetContentType (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:07.455 INFO:teuthology.orchestra.run.smithi033.stderr:testGetOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:testHead (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:testHeadOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:24.448 INFO:teuthology.orchestra.run.smithi033.stderr:testMetadataOnPost (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:38.368 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:45:38.364+0000 37792700 -1 osd.2 185 build_incremental_map_msg missing incremental map 160 2019-12-09T14:45:49.927 INFO:teuthology.orchestra.run.smithi033.stderr:testMetadataOnPut (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:51.001 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:17:12.460 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.017 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:17:12.477 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.065 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:17:12.525 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.067 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:17:12.527 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.090 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:17:12.572 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.091 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:17:12.574 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.250 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:17:12.710 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.276 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:17:12.736 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:51.371 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:17:12.854 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:testNameLimit (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:56.025 INFO:teuthology.orchestra.run.smithi033.stderr:testNoContentLengthForPut (test.functional.tests.TestFile) ... ok 2019-12-09T14:45:56.571 INFO:teuthology.orchestra.run.smithi033.stderr:testPostOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-12-09T14:46:22.311 INFO:teuthology.orchestra.run.smithi033.stderr:testRangedGets (test.functional.tests.TestFile) ... ok 2019-12-09T14:46:32.291 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:32.287+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:32.482 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:32.478+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:33.193 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:33.187+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:33.266 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:33.262+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:33.401 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:33.396+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:33.536 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:33.529+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:34.194 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:34.190+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:34.230 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:34.225+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:34.281 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:34.277+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:34.397 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:34.390+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:34.406 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:34.399+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:34.529 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:34.525+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:35.160 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:35.156+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:35.209 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:35.204+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:35.289 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:35.284+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:35.362 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:35.357+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:35.449 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:35.444+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:35.490 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:35.486+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:36.123 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:36.118+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:36.181 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:36.176+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:36.205 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:36.198+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:36.332 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:36.328+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:36.412 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:36.407+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:36.415 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:36.410+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:36.471 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:36.468+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:37.095 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:37.091+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:37.165 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:37.159+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:37.233 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:37.228+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:37.374 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:37.370+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:37.405 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:37.401+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:37.436 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:37.431+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:37.457 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:37.450+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:38.049 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:38.046+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:38.187 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:38.182+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:38.284 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:38.279+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:38.379 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:38.374+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:38.411 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:38.407+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:38.416 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:38.412+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:38.500 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:38.497+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:39.061 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:39.057+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:39.238 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:39.232+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:39.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:39.298+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:39.367 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:39.362+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:39.412 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:39.407+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:39.429 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:39.425+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:39.539 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:39.535+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:40.063 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:40.059+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:40.236 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:40.230+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:40.322 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:40.317+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:40.399 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:40.394+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:40.426 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:40.421+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:40.478 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:40.474+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:40.580 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:40.576+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:41.110 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:41.107+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:41.208 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:41.204+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:41.305 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:41.299+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:41.385 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:41.379+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:41.397 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:41.392+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:41.531 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:41.526+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:41.631 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:41.628+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:42.136 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:42.127+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:42.202 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:42.196+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:42.341 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:42.336+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:42.371 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:42.365+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:42.408 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:42.402+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:42.551 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:42.546+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:42.617 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:42.612+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:43.147 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:43.143+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:43.222 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:43.217+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:43.344 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:43.339+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:43.371 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:43.366+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:43.410 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:43.406+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:43.533 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:43.529+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:43.586 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:43.581+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:44.187 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:44.181+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:44.190 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:44.186+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:44.385 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:44.380+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:44.399 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:44.394+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:44.434 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:44.428+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:44.495 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:44.492+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:44.638 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:44.634+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:45.183 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:45.179+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:45.195 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:45.191+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:45.382 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:45.377+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:45.403 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:45.397+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:45.433 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:45.428+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:45.504 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:45.499+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:45.674 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:45.669+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:46.165 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:46.160+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:46.193 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:46.189+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:46.369 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:46.364+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:46.387 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:46.383+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:46.408 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:46.403+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:46.548 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:46.544+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:46.633 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:46.628+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:47.213 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:47.207+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:47.243 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:47.239+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:47.350 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:47.344+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:47.361 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:47.355+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:47.372 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:47.366+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:47.522 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:47.518+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:47.595 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:47.591+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:48.255 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:48.247+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:48.265 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:48.260+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:48.356 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:48.349+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:48.366 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:48.360+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:48.372 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:48.366+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:48.502 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:48.497+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:48.574 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:48.570+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:49.240 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:49.235+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:49.288 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:49.282+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:49.341 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:49.335+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:49.382 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:49.376+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:49.408 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:49.402+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:49.514 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:49.510+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:49.582 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:49.578+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:50.286 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:50.282+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:50.304 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:50.297+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:50.316 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:50.310+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:50.402 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:50.395+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:50.453 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:50.447+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:50.552 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:50.549+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:50.613 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:50.608+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:51.264 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:51.260+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:51.287 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:51.281+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:51.307 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:51.301+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:51.382 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:51.376+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:51.428 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:51.422+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:51.553 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:51.549+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:51.636 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:51.632+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:52.248 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:52.241+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:52.259 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:52.255+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:52.317 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:52.311+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:52.372 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:52.366+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:52.425 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:52.419+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:52.541 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:52.536+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:52.607 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:52.603+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:53.214 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:53.210+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:53.280 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:53.274+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:53.300 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:53.293+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:53.354 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:53.348+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:53.467 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:53.461+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:53.572 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:46:53.567+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.198102+0000 front 2019-12-09T14:45:51.935718+0000 (oldest deadline 2019-12-09T14:46:31.501336+0000) 2019-12-09T14:46:53.626 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:53.620+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:54.224 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:54.218+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:54.336 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:54.330+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:54.361 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:54.345+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:54.361 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:54.353+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:46:54.380 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:54.369+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:54.476 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:54.468+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:54.673 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:54.669+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:55.213 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:55.210+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:55.342 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:55.336+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:55.388 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:55.381+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:55.388 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:55.382+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:46:55.420 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:55.415+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:55.515 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:55.509+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:55.645 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:55.642+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:56.195 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:56.191+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:56.315 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:18:17.775 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.327 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:18:17.786 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.335 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:56.330+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:56.338 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:18:17.798 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.342 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:18:17.822 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.343 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:18:17.802 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.347 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:18:17.830 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.400 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:18:17.859 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.407 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:56.400+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:56.407 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:56.402+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:46:56.411 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:18:17.871 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.414 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:18:17.896 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:46:56.442 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:56.428+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:56.511 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:56.505+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:56.679 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:56.675+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:57.190 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:57.185+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:57.325 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:57.319+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:57.398 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:57.392+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:57.404 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:57.396+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:57.404 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:57.397+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:46:57.479 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:57.473+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:57.698 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:57.695+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:58.233 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:58.229+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:58.301 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:58.294+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:58.403 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:58.396+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:58.403 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:58.398+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:46:58.413 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:58.406+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:58.514 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:58.504+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:58.666 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:58.662+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:46:59.208 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:46:59.204+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:46:59.256 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:46:59.249+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:46:59.362 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:59.356+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:46:59.365 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:46:59.360+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:46:59.432 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:46:59.425+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:46:59.547 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:46:59.538+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:46:59.641 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:46:59.637+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:00.186 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:00.181+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:00.247 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:00.240+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:00.392 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:00.386+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:00.394 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:00.389+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:00.456 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:00.451+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:00.505 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:00.494+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:00.656 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:00.652+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:01.236 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:01.231+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:01.262 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:01.253+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:01.389 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:01.384+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:01.390 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:01.384+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:01.459 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:01.452+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:01.485 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:01.480+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:01.665 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:01.661+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:02.248 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:02.242+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:02.280 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:02.276+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:02.378 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:02.373+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:02.380 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:02.375+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:02.489 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:02.483+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:02.508 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:02.502+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:02.665 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:02.661+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:03.220 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:03.213+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:03.331 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:03.327+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:03.375 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:03.368+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:03.376 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:03.370+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:03.526 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:03.520+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:03.539 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:03.524+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:03.667 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:03.662+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:04.208 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:04.203+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:04.302 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:04.298+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:04.383 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:04.377+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:04.385 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:04.380+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:04.550 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:04.545+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:04.571 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:04.566+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:04.660 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:04.657+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:05.188 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:05.180+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:05.306 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:05.301+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:05.352 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:05.347+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:05.356 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:05.348+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:05.537 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:05.532+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:05.570 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:05.565+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:05.668 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:05.664+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:06.190 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:47:06.184+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 46sec e188 v188) 2019-12-09T14:47:06.208 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:06.203+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:06.330 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:06.326+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:06.378 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:06.373+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:06.384 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:06.377+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:06.515 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:06.510+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:06.565 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:06.560+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:06.715 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:06.710+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:07.232 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:07.226+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:07.337 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:07.333+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:07.413 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:07.407+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:07.414 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:07.409+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:07.547 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:07.542+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:07.590 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:07.584+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:07.735 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:07.730+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:08.273 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:08.266+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:08.306 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:08.301+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:08.421 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:08.416+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:08.424 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:08.420+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:08.542 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:08.537+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:08.628 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:08.623+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:08.720 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:08.716+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:09.309 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:09.304+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:09.347 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:09.343+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:09.432 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:09.427+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:09.433 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:09.428+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:09.574 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:09.568+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:09.609 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:09.604+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:09.702 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:09.699+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:10.321 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:10.317+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:10.321 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:10.315+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:10.408 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:10.402+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:50.643715+0000 front 2019-12-09T14:45:51.173964+0000 (oldest deadline 2019-12-09T14:46:33.454352+0000) 2019-12-09T14:47:10.409 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:10.404+0000 1694f700 -1 osd.0 188 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:10.604 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:47:10.599+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:48.548029+0000 front 2019-12-09T14:45:47.971345+0000 (oldest deadline 2019-12-09T14:46:32.601807+0000) 2019-12-09T14:47:10.624 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:47:10.620+0000 1694f700 -1 osd.3 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.519222+0000 front 2019-12-09T14:45:51.555406+0000 (oldest deadline 2019-12-09T14:46:34.324257+0000) 2019-12-09T14:47:10.730 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:47:10.725+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.311965+0000 front 2019-12-09T14:45:51.195879+0000 (oldest deadline 2019-12-09T14:46:32.091958+0000) 2019-12-09T14:47:11.233 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:47:11.229+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 46sec e188 v188) 2019-12-09T14:47:11.315 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:47:11.311+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:51.079255+0000 front 2019-12-09T14:45:50.467780+0000 (oldest deadline 2019-12-09T14:46:35.512510+0000) 2019-12-09T14:47:11.352 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:47:11.348+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:45:46.691236+0000 front 2019-12-09T14:45:47.820564+0000 (oldest deadline 2019-12-09T14:46:32.399982+0000) 2019-12-09T14:47:11.776 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:11.547+0000 1694f700 -1 osd.0 189 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:12.576 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:47:12.566+0000 1694f700 -1 osd.0 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104181 6.3 6:c96ba582:root::.bucket.meta.4a7491a425384668910cd0c74d139441%3ad7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.655:head [create,call version.set in=58b,writefull 0~306 in=306b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e188) 2019-12-09T14:47:21.449 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:18:42.909 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.460 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:18:42.919 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.499 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:18:42.959 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.502 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:18:42.962 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.517 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:18:42.977 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.535 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:18:43.017 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.539 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:18:43.021 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.545 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:18:43.005 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:21.615 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:18:43.097 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:47:35.692 INFO:teuthology.orchestra.run.smithi033.stderr:testSerialization (test.functional.tests.TestFile) ... ok 2019-12-09T14:48:01.385 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:01.378+0000 31786700 -1 osd.5 194 build_incremental_map_msg missing incremental map 166 2019-12-09T14:48:06.581 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:06.578+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:07.555 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:07.552+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:08.553 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:08.550+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:08.858 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:08.853+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:09.523 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:09.518+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:09.524 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:09.520+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:09.837 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:09.832+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:10.322 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:10.316+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:10.543 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:10.537+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:10.546 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:10.542+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:10.828 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:10.822+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:11.229 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:11.223+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:11.229 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:11.223+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:11.291 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:11.286+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:11.506 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:11.500+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:11.507 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:11.503+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:11.856 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:11.851+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:12.272 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:12.265+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:12.276 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:12.271+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:12.313 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:12.307+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:12.559 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:12.555+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:12.560 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:12.557+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:12.628 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:12.622+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:12.870 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:12.866+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:13.292 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:13.286+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:13.295 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:13.292+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:13.323 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:13.319+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:13.593 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:13.588+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:13.594 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:13.591+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:13.626 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:13.621+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:13.881 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:13.878+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:14.249 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:14.246+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:14.298 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:14.292+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:14.348 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:14.343+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:14.638 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:14.634+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:14.638 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:14.636+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:14.647 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:14.641+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:14.905 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:14.900+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:15.258 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:15.253+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:15.287 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:15.283+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:15.356 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:15.351+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:15.609 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:15.604+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:15.643 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:15.638+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:15.644 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:15.641+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:15.938 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:15.934+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:16.220 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:16.215+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:16.256 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:16.251+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:16.372 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:16.366+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:16.647 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:16.643+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:16.648 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:16.645+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:16.662 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:16.656+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:16.896 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:16.893+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:17.268 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:17.263+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:17.275 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:17.272+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:17.403 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:17.396+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:17.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:17.662+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:17.669 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:17.665+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:17.670 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:17.667+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:17.947 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:17.943+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:18.253 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:18.247+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:18.257 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:18.254+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:18.436 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:18.431+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:18.645 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:18.639+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:18.660 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:18.657+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:18.664 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:18.662+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:18.916 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:18.910+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:19.224 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:19.219+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:19.297 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:19.291+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:19.483 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:19.479+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:19.649 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:19.644+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:19.651 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:19.647+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:19.655 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:19.652+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:19.963 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:19.955+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:20.190 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:20.184+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:20.305 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:20.301+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:20.500 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:20.496+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:20.682 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:20.676+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:20.696 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:20.691+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:20.698 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:20.695+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:20.939 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:20.934+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:21.223 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:21.218+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:21.357 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:21.352+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:21.526 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:21.521+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:21.696 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:21.691+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:21.697 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:21.693+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:21.708 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:21.703+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:21.988 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:21.983+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:22.275 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:22.269+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:22.403 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:22.397+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:22.513 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:22.507+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:22.671 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:22.665+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:22.678 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:22.675+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:22.742 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:22.736+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:23.044 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:23.040+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:23.296 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:23.290+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:23.455 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:23.450+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:23.556 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:23.550+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:23.698 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:23.692+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:23.709 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:23.703+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:23.710 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:23.706+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:24.080 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:24.074+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:24.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:24.298+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:24.460 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:24.455+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:24.552 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:24.547+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:24.693 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:24.687+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:24.694 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:24.690+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:24.737 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:24.731+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:25.114 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:25.110+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:25.331 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:25.318+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:25.453 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:25.448+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:25.568 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:25.562+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:25.683 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:25.674+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:25.686 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:25.677+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:25.718 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:25.712+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:26.160 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:26.156+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:26.382 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:26.375+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:26.523 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:26.501+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:26.571 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:26.565+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:26.714 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:26.710+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:26.716 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:26.711+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:26.717 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:26.714+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:27.162 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:27.158+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:27.426 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:27.420+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:27.479 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:27.474+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:27.585 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:27.578+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:27.655 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:27.651+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:27.702 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:27.697+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:27.754 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:27.749+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:27.755 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:27.751+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:28.165 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:28.159+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:28.474 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:28.469+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:28.522 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:28.517+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:28.634 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:28.628+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:28.702 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:28.698+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:28.722 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:28.716+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:28.756 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:28.751+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:28.760 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:28.757+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:29.186 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:29.181+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:29.463 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:29.458+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:29.527 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:29.522+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:29.672 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:29.667+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:29.702 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:29.698+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:29.723 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:29.718+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:29.787 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:29.782+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:29.790 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:29.787+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:30.146 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:30.141+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:30.508 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:30.501+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:30.538 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:30.530+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:30.711 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:30.706+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:30.736 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:30.731+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:30.756 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:30.749+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:30.768 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:30.763+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:30.768 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:30.765+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:31.111 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:31.106+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:31.504 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:31.499+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:31.553 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:31.549+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:31.682 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:31.677+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:31.725 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:31.720+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:31.726 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:31.722+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:31.733 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:31.729+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:31.755 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:31.752+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:32.149 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:32.145+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:32.486 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:32.480+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:32.540 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:32.535+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:32.650 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:32.644+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:32.700 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:32.694+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:32.701 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:32.697+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:32.723 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:32.719+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:32.755 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:32.750+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:33.164 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:33.158+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:33.480 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:33.476+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:33.575 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:33.570+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:33.624 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:33.618+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:33.725 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:33.721+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:33.728 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:33.725+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:33.775 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:33.761+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:33.782 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:33.779+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:33.805 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:33.799+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:34.172 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:34.168+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:34.440 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:34.434+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:34.535 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:34.530+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:34.605 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:34.595+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:34.682 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:34.678+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:34.686 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:34.683+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:34.784 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:34.777+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:34.785 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:34.780+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:34.799 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:34.793+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:35.172 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:35.167+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:35.423 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:35.417+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:35.503 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:35.498+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:35.576 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:35.571+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:35.712 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:35.707+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:35.715 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:35.712+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:35.775 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:35.770+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:35.779 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:35.776+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:35.805 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:35.800+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:36.139 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:36.133+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:36.408 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:36.403+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:36.516 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:36.511+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:36.630 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:36.624+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:36.710 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:36.705+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:36.710 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:36.707+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:36.822 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:36.818+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:36.826 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:36.822+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:36.833 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:36.827+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:37.159 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:37.155+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:37.454 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:37.447+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:37.566 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:37.561+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:37.646 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:37.642+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:37.708 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:37.704+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:37.712 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:37.709+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:37.805 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:37.787+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:37.819 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:37.812+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:37.819 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:37.811+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:38.117 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:38.112+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:38.482 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:38.476+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:38.564 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:38.560+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:38.681 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:38.675+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:38.741 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:38.736+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:38.745 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:38.742+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:38.855 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:38.849+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:38.855 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:38.851+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:38.870 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:38.865+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:39.126 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:39.121+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:39.501 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:39.495+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:39.599 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:39.594+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:39.682 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:39.677+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:39.799 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:39.793+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:39.800 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:39.796+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:39.846 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:39.839+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:39.889 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:39.885+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:39.896 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:39.890+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:40.145 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:40.140+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:40.528 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:40.523+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:40.557 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:40.552+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:40.699 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:40.694+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:40.801 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:40.796+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:40.802 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:40.799+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:40.812 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:40.806+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:40.910 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:40.906+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:40.913 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:40.910+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:41.195 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:41.188+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:41.567 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:41.561+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:41.589 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:41.584+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:41.626 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:48:41.619+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e194 v194) 2019-12-09T14:48:41.724 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:41.720+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:41.794 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:41.788+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:41.797 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:41.791+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:41.803 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:41.798+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:41.890 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:41.884+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:41.891 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:41.887+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:42.221 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:42.216+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:42.535 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:42.530+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:42.587 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:42.581+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:42.741 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:42.735+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:42.771 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:42.765+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:42.772 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:42.768+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:42.784 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:42.777+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:42.853 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:42.848+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:42.857 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:42.853+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:43.248 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:43.243+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:43.562 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:43.556+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:43.563 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:43.557+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:43.781 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:43.776+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:43.783 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:43.778+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:43.823 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:43.818+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:43.826 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:43.823+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:43.861 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:43.855+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:43.862 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:43.858+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:44.236 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:44.230+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:44.552 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:44.546+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:44.597 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:44.592+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:44.803 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:44.798+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:44.831 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:44.826+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:44.835 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:44.829+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:44.835 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:44.832+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:44.855 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:44.849+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:44.858 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:44.852+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:45.277 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:45.272+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:45.532 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:45.524+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:45.635 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:45.629+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:45.762 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:45.758+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:45.803 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:45.797+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:45.865 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:45.860+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:45.865 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:45.861+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:45.873 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:45.869+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:45.876 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:45.873+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:46.309 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:46.304+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:46.522 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:46.517+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:46.673 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:46.669+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:46.700 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:48:46.695+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e194 v194) 2019-12-09T14:48:46.801 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:46.796+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:46.811 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:46.806+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:46.826 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:46.822+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:46.827 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:46.823+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:46.920 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:46.913+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:46.921 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:46.914+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:47.287 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:47.283+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:47.513 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:47.502+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:47.661 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:47.656+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:47.812 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:47.807+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:47.813 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:47.809+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:47.847 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:47.841+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:47.862 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:47.856+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:47.889 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:47.884+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:47.892 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:47.889+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:48.254 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:48.248+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:48.475 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:48.468+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:48.636 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:48.631+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:48.822 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:48.815+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:48.830 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:48.825+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:48.833 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:48.830+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:48.890 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:48.878+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:48.890 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:48.881+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:48.891 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:48.884+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:49.300 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:49.295+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:49.476 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:49.469+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:49.683 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:49.677+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:49.821 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:49.815+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:49.822 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:49.818+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:49.861 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:49.855+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:49.887 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:49.881+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:49.919 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:49.914+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:49.920 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:49.916+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:50.344 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:50.339+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:50.486 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:50.479+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:50.721 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:50.716+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:50.803 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:50.798+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:50.805 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:50.800+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:50.887 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:50.881+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:50.898 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:50.893+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:50.923 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:50.917+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:50.924 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:50.920+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:51.369 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:51.365+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:51.517 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:51.510+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:51.715 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:48:51.709+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e194 v194) 2019-12-09T14:48:51.781 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:51.761+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:51.800 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:51.795+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:51.801 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:51.798+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:51.882 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:51.876+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:51.939 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:51.933+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:51.946 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:51.942+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:51.950 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:51.947+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:52.372 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:52.367+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:52.488 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:52.482+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:52.753 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:52.747+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:52.773 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:52.766+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:52.774 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:52.769+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:52.835 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:52.829+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:52.908 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:52.903+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:52.909 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:52.905+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:52.917 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:52.912+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:53.384 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:53.379+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:53.479 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:53.473+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:53.733 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:53.729+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:53.779 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:53.774+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:53.780 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:53.776+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:53.850 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:53.844+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:53.953 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:53.947+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:53.953 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:53.950+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:53.962 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:53.956+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:54.395 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:54.389+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:54.457 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:54.450+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:54.750 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:54.744+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:54.751 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:54.747+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:54.756 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:54.751+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:54.815 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:54.809+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:54.927 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:54.922+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:54.928 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:54.924+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:54.954 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:54.947+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:55.379 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:55.375+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:55.505 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:55.498+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:55.745 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:55.737+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:55.789 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:55.783+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:55.790 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:55.786+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:55.800 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:55.794+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:55.914 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:55.910+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:55.917 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:55.913+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:55.979 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:55.973+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:56.427 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:56.416+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:56.464 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:56.458+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:56.730 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:48:56.725+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e194 v194) 2019-12-09T14:48:56.781 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:56.772+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:56.819 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:56.815+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:56.822 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:56.819+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:56.826 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:56.819+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:56.931 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:56.927+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:56.932 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:56.928+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:57.026 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:57.020+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:57.399 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:57.394+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:57.432 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:57.424+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:57.784 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:57.779+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:57.788 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:57.785+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:57.800 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:57.792+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:57.830 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:57.825+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:57.906 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:57.901+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:57.907 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:57.903+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:57.980 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:57.973+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:58.373 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:58.368+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:58.472 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:58.464+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:58.779 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:58.773+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:58.780 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:58.776+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:58.825 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:58.820+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:58.826 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:58.818+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:58.911 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:58.905+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:58.912 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:58.908+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:58.948 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:58.942+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:48:59.426 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:48:59.421+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:48:59.440 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:48:59.434+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:48:59.796 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:59.791+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:48:59.799 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:48:59.796+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:48:59.802 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:48:59.797+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:48:59.814 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:48:59.807+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:48:59.864 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:59.860+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:48:59.867 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:48:59.865+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:48:59.993 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:48:59.985+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:00.434 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:00.428+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:00.473 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:00.468+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:00.771 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:00.765+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:00.783 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:00.777+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:00.803 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:00.799+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:00.817 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:00.808+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:00.903 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:00.898+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:00.904 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:00.900+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:01.025 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:01.011+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:01.439 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:01.432+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:01.512 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:01.507+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:01.743 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:49:01.738+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e194 v194) 2019-12-09T14:49:01.756 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:01.746+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:01.785 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:01.780+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:01.786 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:01.782+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:01.803 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:01.799+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:01.891 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:01.887+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:01.894 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:01.891+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:01.995 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:01.989+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:02.467 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:02.460+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:02.529 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:02.519+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:02.788 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:02.783+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:02.789 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:02.785+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:02.801 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:02.796+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:02.823 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:02.819+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:02.910 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:02.905+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:02.911 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:02.907+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:02.999 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:02.994+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:03.425 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:03.417+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:03.575 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:03.570+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:03.744 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:03.738+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:03.745 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:03.741+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:03.808 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:03.801+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:03.845 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:03.841+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:03.921 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:03.915+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:03.922 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:03.918+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:04.014 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:04.009+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:04.429 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:04.423+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:04.620 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:04.607+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:04.729 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:04.724+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:04.731 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:04.726+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:04.782 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:04.777+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:04.803 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:04.799+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:04.965 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:04.960+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:04.967 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:04.965+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:04.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:04.971+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:05.414 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:05.409+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:05.619 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:05.611+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:05.743 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:05.739+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:05.746 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:05.743+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:05.751 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:05.747+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:05.857 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:05.852+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:05.944 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:05.939+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:05.945 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:05.941+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:05.961 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:05.956+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:06.390 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:06.384+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:06.644 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:06.639+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:06.738 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:06.733+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:06.739 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:06.736+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:06.754 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:49:06.749+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e194 v194) 2019-12-09T14:49:06.771 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:06.766+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:06.885 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:06.881+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:06.922 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:06.919+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:06.923 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:06.920+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:06.978 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:06.972+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:07.435 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:07.430+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:07.634 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:07.625+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:07.735 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:07.731+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:07.736 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:07.733+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:07.736 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:07.731+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:07.853 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:07.845+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:07.886 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:07.882+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:07.887 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:07.885+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:08.006 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:08.000+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:08.392 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:08.386+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:08.673 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:08.669+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:08.696 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:08.690+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:08.728 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:08.724+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:08.731 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:08.729+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:08.829 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:08.825+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:08.924 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:08.921+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:08.927 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:08.925+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:09.034 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:09.029+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:09.368 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:09.363+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:09.701 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:09.695+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:09.724 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:09.720+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:09.733 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:09.729+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:09.734 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:09.731+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:09.863 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:09.858+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:09.957 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:09.953+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:09.958 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:09.955+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:10.065 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:10.060+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:10.382 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:10.376+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:10.663 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:10.657+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:10.730 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:10.725+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:10.780 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:10.773+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:10.780 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:10.776+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:10.852 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:10.849+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:10.987 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:10.983+0000 1694f700 -1 osd.4 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.145979+0000 front 2019-12-09T14:47:23.263566+0000 (oldest deadline 2019-12-09T14:48:09.023928+0000) 2019-12-09T14:49:10.988 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:10.985+0000 1694f700 -1 osd.4 194 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:11.032 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:49:11.026+0000 1694f700 -1 osd.3 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:23.544714+0000 front 2019-12-09T14:47:23.680400+0000 (oldest deadline 2019-12-09T14:48:09.423909+0000) 2019-12-09T14:49:11.352 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:49:11.344+0000 1694f700 -1 osd.0 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.161508+0000 front 2019-12-09T14:47:27.195761+0000 (oldest deadline 2019-12-09T14:48:10.606543+0000) 2019-12-09T14:49:11.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:49:11.648+0000 1694f700 -1 osd.1 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.997761+0000 front 2019-12-09T14:47:27.994070+0000 (oldest deadline 2019-12-09T14:48:11.671068+0000) 2019-12-09T14:49:11.782 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:49:11.779+0000 1694f700 -1 osd.6 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:24.050935+0000 front 2019-12-09T14:47:24.078825+0000 (oldest deadline 2019-12-09T14:48:08.107854+0000) 2019-12-09T14:49:11.795 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:11.790+0000 1694f700 -1 osd.5 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:47:53.032913+0000 (oldest deadline 2019-12-09T14:48:33.032913+0000) 2019-12-09T14:49:11.796 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:49:11.793+0000 1694f700 -1 osd.5 194 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e192) 2019-12-09T14:49:11.805 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:49:11.798+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e194 v194) 2019-12-09T14:49:11.889 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:49:11.880+0000 1694f700 -1 osd.7 194 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:47:27.026698+0000 front 2019-12-09T14:47:27.094193+0000 (oldest deadline 2019-12-09T14:48:10.500312+0000) 2019-12-09T14:49:12.456 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:12.451+0000 1694f700 -1 osd.4 195 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:49:13.456 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:49:13.444+0000 1694f700 -1 osd.4 196 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104337 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] ondisk+write+known_if_redirected e190) 2019-12-09T14:50:26.041 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:26.034+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:26.189 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:26.183+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:27.023 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:27.019+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:27.224 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:27.221+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:27.309 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:27.305+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:27.406 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:27.402+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:27.907 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:27.902+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:28.015 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:28.010+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:28.218 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:28.215+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:28.336 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:28.328+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:28.440 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:28.437+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:28.503 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:28.499+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:28.745 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:28.740+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:28.933 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:28.926+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:28.995 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:28.989+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:29.229 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:29.224+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:29.345 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:29.341+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:29.475 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:29.471+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:29.487 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:29.483+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:29.763 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:29.758+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:29.920 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:29.915+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:29.979 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:29.974+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:30.220 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:30.216+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:30.332 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:30.328+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:30.451 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:30.444+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:30.484 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:30.478+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:30.767 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:30.760+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:30.939 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:30.933+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:30.973 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:30.968+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:31.214 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:31.210+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:31.296 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:31.293+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:31.403 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:31.400+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:31.509 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:31.503+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:31.761 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:31.756+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:31.910 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:31.904+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:32.019 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:32.013+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:32.230 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:32.226+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:32.346 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:32.341+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:32.439 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:32.435+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:32.490 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:32.481+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:32.752 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:32.747+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:32.924 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:32.918+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:32.983 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:32.978+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:33.247 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:33.243+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:33.320 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:33.315+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:33.487 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:33.482+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:33.507 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:33.502+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:33.726 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:33.720+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:33.919 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:33.914+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:34.024 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:34.019+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:34.242 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:34.238+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:34.322 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:34.318+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:34.475 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:34.469+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:34.513 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:34.509+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:34.739 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:34.732+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:34.964 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:34.959+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:35.036 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:35.031+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:35.268 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:35.264+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:35.351 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:35.347+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:35.472 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:35.469+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:35.501 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:35.495+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:35.742 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:35.736+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:35.998 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:35.989+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:36.014 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:36.010+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:36.280 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:36.275+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:36.310 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:36.307+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:36.481 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:36.476+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:36.491 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:36.486+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:36.775 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:36.770+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:36.960 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:36.956+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:37.068 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:37.062+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:37.294 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:37.290+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:37.328 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:37.323+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:37.451 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:37.445+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:37.533 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:37.528+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:37.778 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:37.772+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:37.955 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:37.950+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:38.036 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:38.032+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:38.303 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:38.299+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:38.307 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:38.302+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:38.444 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:38.438+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:38.582 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:38.579+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:38.741 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:38.736+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:38.955 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:38.951+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:39.073 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:39.067+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:39.308 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:39.303+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:39.349 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:39.345+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:39.421 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:39.415+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:39.616 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:39.610+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:39.782 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:39.776+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:39.999 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:39.994+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:40.032 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:40.027+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:40.326 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:40.322+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:40.378 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:40.373+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:40.409 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:40.403+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:40.580 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:40.576+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:40.767 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:40.761+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:41.028 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:41.022+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:41.058 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:41.053+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:41.344 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:41.339+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:41.358 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:41.354+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:41.409 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:41.404+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:41.552 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:41.548+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:41.820 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:41.814+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:41.992 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:41.986+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:42.051 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:42.046+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:42.337 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:42.333+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:42.347 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:42.342+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:42.372 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:42.366+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:42.588 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:42.583+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:42.861 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:42.855+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:43.013 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:43.007+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:43.069 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:43.063+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:43.323 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:43.319+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:43.335 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:43.329+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:43.340 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:43.335+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:43.613 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:43.609+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:43.889 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:43.883+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:43.990 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:43.985+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:44.063 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:44.058+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:44.280 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:44.275+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:44.364 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:44.360+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:44.377 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:44.372+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:44.587 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:44.581+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:44.924 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:44.919+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:44.985 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:44.981+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:45.011 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:22:06.471 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.019 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:22:06.479 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.026 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:22:06.508 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.031 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:22:06.514 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.035 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:22:06.495 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.039 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:22:06.499 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.104 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:45.099+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:45.260 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:22:06.720 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.268 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:22:06.751 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.277 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:45.273+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:45.318 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:22:06.778 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:45.328 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:45.325+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:45.407 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:45.402+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:45.625 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:45.621+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:45.895 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:45.890+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:46.036 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:46.031+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:46.114 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:46.109+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:46.304 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:46.299+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:46.323 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:46.319+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:46.420 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:46.415+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:46.642 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:46.638+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:46.881 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:46.876+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:46.997 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:46.990+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:47.099 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:47.091+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:47.287 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:47.283+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:47.348 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:47.345+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:47.418 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:47.413+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:47.664 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:47.660+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:47.912 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:47.904+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:47.960 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:47.954+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:48.109 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:48.097+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:48.267 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:48.263+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:48.314 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:48.311+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:48.392 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:48.386+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:48.705 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:48.702+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:48.921 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:48.914+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:49.001 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:48.996+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:49.125 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:49.118+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:49.231 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:49.227+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:49.303 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:49.299+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:49.378 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:49.371+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:49.727 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:49.723+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:49.875 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:49.868+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:49.965 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:49.960+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:50.127 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:50.121+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:50.214 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:50.211+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:50.344 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:50.339+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:50.356 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:50.348+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:50.701 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:50.696+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:50.832 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:50.826+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:50.973 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:50.966+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:51.134 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:51.126+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:51.213 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:51.209+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:51.328 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:51.324+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:51.373 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:51.367+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:51.751 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:51.747+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:51.849 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:51.843+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:51.989 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:51.982+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:52.162 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:52.156+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:52.229 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:52.226+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:52.305 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:52.300+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:52.343 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:52.337+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:52.711 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:52.707+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:52.844 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:52.838+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:53.018 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:53.012+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:53.158 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:53.151+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:53.250 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:53.246+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:53.313 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:53.308+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:53.326 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:53.321+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:53.691 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:53.687+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:53.885 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:53.877+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:54.021 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:54.015+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:54.203 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:54.197+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:54.290 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:54.284+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:54.331 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:54.325+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:54.361 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:54.358+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:54.702 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:54.700+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:54.917 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:54.912+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:55.040 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:55.035+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:55.175 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:55.169+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:55.335 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:55.331+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:55.376 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:55.370+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:55.395 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:55.392+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:55.704 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:55.701+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:55.884 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:55.875+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:56.017 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:56.010+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:56.154 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:56.147+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:56.330 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:56.327+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:56.360 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:56.354+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:56.403 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:56.400+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:56.720 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:56.717+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:56.918 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:56.912+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:57.069 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:57.061+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:57.158 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:50:57.153+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e199 v199) 2019-12-09T14:50:57.189 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:57.182+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:57.335 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:57.334+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:57.387 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:57.377+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:57.409 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:57.406+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:57.694 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:57.689+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:57.929 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:57.923+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:58.050 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:58.043+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:58.241 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:58.234+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:58.325 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:58.322+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:58.406 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:58.400+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:58.435 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:58.432+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:58.706 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:58.703+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:58.888 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:58.881+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:50:59.043 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:50:59.037+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:50:59.260 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:50:59.254+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:50:59.294 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:50:59.291+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:50:59.394 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:50:59.387+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:50:59.440 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:50:59.436+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:50:59.714 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:50:59.711+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:50:59.882 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:50:59.875+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:51:00.086 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:51:00.080+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:51:00.274 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:00.270+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:51:00.293 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:51:00.287+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:51:00.409 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:00.406+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:51:00.431 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:51:00.425+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:51:00.754 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:51:00.750+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:51:00.842 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:51:00.833+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:51:01.120 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:51:01.114+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:51:01.298 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:51:01.292+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:51:01.307 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:01.302+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:51:01.430 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:51:01.423+0000 1694f700 -1 osd.2 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.420446+0000 front 2019-12-09T14:49:44.503559+0000 (oldest deadline 2019-12-09T14:50:27.743926+0000) 2019-12-09T14:51:01.431 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:01.427+0000 1694f700 -1 osd.4 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:42.273501+0000 front 2019-12-09T14:49:42.268527+0000 (oldest deadline 2019-12-09T14:50:25.453269+0000) 2019-12-09T14:51:01.715 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:51:01.710+0000 1694f700 -1 osd.6 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:45.075900+0000 front 2019-12-09T14:49:44.886450+0000 (oldest deadline 2019-12-09T14:50:26.510878+0000) 2019-12-09T14:51:01.828 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:51:01.822+0000 1694f700 -1 osd.3 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:44.343323+0000 front 2019-12-09T14:49:44.345494+0000 (oldest deadline 2019-12-09T14:50:28.288792+0000) 2019-12-09T14:51:02.145 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:51:02.136+0000 1694f700 -1 osd.0 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.809124+0000 front 2019-12-09T14:49:43.615917+0000 (oldest deadline 2019-12-09T14:50:27.008292+0000) 2019-12-09T14:51:02.236 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:51:02.226+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e199 v199) 2019-12-09T14:51:02.263 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:02.260+0000 1694f700 -1 osd.7 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.627703+0000 front 2019-12-09T14:49:43.631687+0000 (oldest deadline 2019-12-09T14:50:26.838043+0000) 2019-12-09T14:51:02.314 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:51:02.308+0000 1694f700 -1 osd.1 199 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:49:43.766535+0000 front 2019-12-09T14:49:43.840976+0000 (oldest deadline 2019-12-09T14:50:25.952909+0000) 2019-12-09T14:51:53.788 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:53.782+0000 31786700 -1 osd.4 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:53.809 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:51:53.799+0000 31f87700 -1 osd.1 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:53.831 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:53.811+0000 36790700 -1 osd.4 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:53.899 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:51:53.879+0000 32182700 -1 osd.6 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:53.943 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:53.882+0000 37f93700 -1 osd.4 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:53.972 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:51:53.952+0000 35989700 -1 osd.6 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.043 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:53.986+0000 31786700 -1 osd.4 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.129 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:51:54.122+0000 31786700 -1 osd.1 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.137 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:54.071+0000 37792700 -1 osd.4 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.164 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:54.152+0000 35f8f700 -1 osd.7 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.216 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:51:54.147+0000 32983700 -1 osd.6 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.272 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:51:54.222+0000 32f89700 -1 osd.1 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.333 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:51:54.262+0000 36790700 -1 osd.1 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.351 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:51:54.136+0000 34f8d700 -1 osd.3 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.415 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:54.368+0000 1694f700 -1 osd.4 204 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:51:54.422 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:54.393+0000 36790700 -1 osd.7 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.436 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:51:54.402+0000 3818e700 -1 osd.6 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.452 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:54.445+0000 37792700 -1 osd.7 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.543 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:51:54.398+0000 31f87700 -1 osd.3 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.585 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:54.580+0000 36790700 -1 osd.7 204 build_incremental_map_msg missing incremental map 204 2019-12-09T14:51:54.587 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:51:54.473+0000 32788700 -1 osd.2 203 build_incremental_map_msg missing incremental map 203 2019-12-09T14:51:54.595 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:54.589+0000 37792700 -1 osd.7 204 build_incremental_map_msg missing incremental map 204 2019-12-09T14:51:54.635 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:54.619+0000 31786700 -1 osd.7 204 build_incremental_map_msg missing incremental map 204 2019-12-09T14:51:54.885 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:54.880+0000 31f87700 -1 osd.7 205 build_incremental_map_msg missing incremental map 204 2019-12-09T14:51:55.326 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:55.323+0000 1694f700 -1 osd.4 205 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:51:56.252 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:51:56.249+0000 37f93700 -1 osd.7 206 build_incremental_map_msg missing incremental map 204 2019-12-09T14:51:56.301 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:56.299+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:51:57.343 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:57.338+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:51:58.341 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:58.338+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:51:59.360 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:51:59.357+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:00.393 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:00.387+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:00.394 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:00.390+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:01.043 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:01.036+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:01.153 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:01.147+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:01.429 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:01.421+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:01.429 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:01.424+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:01.489 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:01.483+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:02.061 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:02.056+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:02.091 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:02.086+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:02.151 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:02.146+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:02.452 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:02.445+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:02.453 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:02.448+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:02.512 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:02.506+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:03.052 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:03.047+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:03.079 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:03.073+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:03.175 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:03.168+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:03.489 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:03.483+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:03.493 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:03.490+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:03.540 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:03.534+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:04.009 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:04.005+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:04.098 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:04.092+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:04.175 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:04.170+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:04.504 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:04.499+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:04.505 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:04.502+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:04.562 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:04.557+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:05.046 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:05.041+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:05.062 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:05.056+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:05.154 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:05.149+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:05.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:05.488+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:05.495 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:05.491+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:05.527 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:05.520+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:06.069 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:06.064+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:06.074 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:06.069+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:06.165 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:06.159+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:06.470 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:06.466+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:06.474 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:06.470+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:06.564 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:06.559+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:07.083 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:07.078+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:07.092 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:07.088+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:07.197 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:07.191+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:07.483 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:07.478+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:07.487 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:07.483+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:07.600 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:07.595+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:08.042 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:08.037+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:08.145 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:08.140+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:08.246 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:08.240+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:08.468 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:08.462+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:08.469 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:08.465+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:08.556 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:08.550+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:09.058 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:09.051+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:09.165 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:09.160+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:09.205 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:09.200+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:09.434 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:09.424+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:09.438 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:09.431+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:09.516 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:09.510+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:10.029 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:10.023+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:10.170 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:10.164+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:10.208 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:10.202+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:10.410 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:10.404+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:10.411 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:10.407+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:10.556 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:10.550+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:10.992 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:10.987+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:11.171 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:11.167+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:11.219 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:11.213+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:11.370 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:11.366+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:11.371 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:11.368+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:11.588 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:11.581+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:12.026 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:12.021+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:12.166 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:12.161+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:12.239 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:12.232+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:12.382 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:12.377+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:12.383 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:12.379+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:12.617 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:12.610+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:13.076 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:13.070+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:13.162 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:13.157+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:13.251 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:13.244+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:13.352 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:13.346+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:13.353 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:13.349+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:13.596 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:13.591+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:14.122 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:14.116+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:14.218 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:14.212+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:14.296 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:14.291+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:14.349 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:14.342+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:14.349 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:14.345+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:14.637 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:14.631+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:15.154 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:15.149+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:15.172 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:15.167+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:15.328 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:15.323+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:15.365 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:15.359+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:15.368 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:15.365+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:15.682 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:15.676+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:16.155 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:16.148+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:16.170 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:16.164+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:16.313 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:16.308+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:16.338 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:16.332+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:16.339 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:16.335+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:16.674 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:16.668+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:17.178 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:17.173+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:17.184 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:17.179+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:17.282 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:17.277+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:17.318 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:17.312+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:17.318 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:17.314+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:17.677 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:17.672+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:18.153 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:18.147+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:18.155 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:18.151+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:18.309 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:18.302+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:18.337 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:18.332+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:18.341 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:18.338+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:18.695 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:18.689+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:19.119 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:19.113+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:19.197 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:19.191+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:19.278 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:19.272+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:19.334 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:19.328+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:19.335 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:19.331+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:19.706 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:19.702+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:20.095 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:20.090+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:20.208 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:20.203+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:20.320 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:20.313+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:20.380 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:20.375+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:20.381 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:20.377+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:20.717 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:20.712+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:21.139 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:21.133+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:21.195 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:21.190+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:21.276 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:21.271+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:21.384 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:21.380+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:21.388 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:21.384+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:21.707 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:21.702+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:22.152 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:22.148+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:22.169 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:22.162+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:22.265 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:22.258+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:22.361 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:22.352+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:22.378 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:22.355+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:22.753 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:22.748+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:23.172 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:23.166+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:23.196 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:23.190+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:23.300 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:23.294+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:23.370 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:23.365+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:23.371 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:23.367+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:23.762 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:23.756+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:24.172 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:24.168+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:24.182 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:24.177+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:24.291 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:24.286+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:24.379 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:24.372+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:24.380 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:24.376+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:24.734 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:24.727+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:25.008 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:52:25.005+0000 1694f700 -1 osd.5 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e203) 2019-12-09T14:52:25.138 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:25.133+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:25.190 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:25.184+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:25.255 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:25.250+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:25.350 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:25.345+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:25.354 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:25.351+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:25.736 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:25.729+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:25.993 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:52:25.989+0000 1694f700 -1 osd.5 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e203) 2019-12-09T14:52:26.125 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:26.120+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:26.146 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:26.140+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:26.211 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:26.204+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:26.371 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:26.365+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:26.372 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:26.368+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:26.763 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:52:26.757+0000 1694f700 -1 osd.3 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.203960+0000 front 2019-12-09T14:51:18.409628+0000 (oldest deadline 2019-12-09T14:52:00.477765+0000) 2019-12-09T14:52:26.957 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:52:26.954+0000 1694f700 -1 osd.5 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e203) 2019-12-09T14:52:27.107 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:52:27.101+0000 1694f700 -1 osd.1 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.026739+0000 front 2019-12-09T14:51:17.976574+0000 (oldest deadline 2019-12-09T14:52:00.266982+0000) 2019-12-09T14:52:27.127 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:52:27.121+0000 1694f700 -1 osd.6 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:19.893273+0000 front 2019-12-09T14:51:20.053045+0000 (oldest deadline 2019-12-09T14:52:01.890967+0000) 2019-12-09T14:52:27.214 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:27.208+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:27.409 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:27.405+0000 1694f700 -1 osd.4 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:15.935108+0000 front 2019-12-09T14:51:16.005583+0000 (oldest deadline 2019-12-09T14:52:00.026695+0000) 2019-12-09T14:52:27.413 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:27.410+0000 1694f700 -1 osd.4 206 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-12-09T14:52:28.216 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:52:28.211+0000 1694f700 -1 osd.0 206 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:51:18.825597+0000 front 2019-12-09T14:51:18.674676+0000 (oldest deadline 2019-12-09T14:52:00.322397+0000) 2019-12-09T14:52:28.392 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:28.390+0000 1694f700 -1 osd.4 207 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:104794 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=340b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e201) 2019-12-09T14:52:29.428 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:52:29.417+0000 1694f700 -1 osd.4 208 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:104797 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=175b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e201) 2019-12-09T14:52:57.088 INFO:teuthology.orchestra.run.smithi033.stderr:testStackedOverwrite (test.functional.tests.TestFile) ... ok 2019-12-09T14:52:57.113 INFO:teuthology.orchestra.run.smithi033.stderr:testTooLongName (test.functional.tests.TestFile) ... ok 2019-12-09T14:52:59.864 INFO:teuthology.orchestra.run.smithi033.stderr:testZeroByteFile (test.functional.tests.TestFile) ... ok 2019-12-09T14:53:53.007 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:53.002+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:53:54.047 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:54.042+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:53:55.008 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:55.003+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:53:56.018 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:56.012+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:53:57.040 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:57.035+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:53:58.034 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:58.029+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:53:59.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:59.007+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:53:59.972 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:53:59.966+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:01.023 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:01.017+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:02.025 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:02.019+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:02.992 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:02.987+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:04.008 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:04.004+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:05.059 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:05.054+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:06.084 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:06.074+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:07.078 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:07.074+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:08.111 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:08.106+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:09.112 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:09.106+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:10.103 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:10.099+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:11.098 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:11.093+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:12.089 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:12.084+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:13.075 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:13.071+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:14.063 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:14.058+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:15.095 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:15.091+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:16.093 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:16.088+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:17.128 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:17.123+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:17.128 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:17.124+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:17.785 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:17.779+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:18.141 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:18.136+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:18.141 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:18.137+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:18.776 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:18.772+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:18.850 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:18.845+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:18.855 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:18.850+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:19.052 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:19.047+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:19.147 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:19.141+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:19.150 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:19.145+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:19.739 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:19.735+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:19.866 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:19.861+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:19.899 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:19.894+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:20.024 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:20.018+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:20.177 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:20.169+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:20.177 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:20.172+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:20.180 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:20.174+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:20.403 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:20.398+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:20.767 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:20.762+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:20.857 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:20.852+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:20.936 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:20.931+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:21.028 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:21.023+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:21.175 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:21.170+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:21.197 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:21.189+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:21.197 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:21.192+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:21.443 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:21.438+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:21.748 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:21.744+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:21.901 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:21.897+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:21.922 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:21.917+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:21.985 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:21.977+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:22.135 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:22.130+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:22.186 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:22.179+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:22.187 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:22.181+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:22.496 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:22.491+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:22.787 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:22.781+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:22.907 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:22.900+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:22.929 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:22.923+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:22.954 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:22.949+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:23.173 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:23.168+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:23.197 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:23.191+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:23.198 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:23.192+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:23.482 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:23.476+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:23.749 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:23.745+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:23.945 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:23.940+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:23.955 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:23.951+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:24.003 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:23.998+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:24.154 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:24.148+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:24.234 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:24.229+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:24.236 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:24.232+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:24.512 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:24.507+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:24.742 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:24.738+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:24.982 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:24.975+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:24.990 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:24.986+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:25.003 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:24.998+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:25.128 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:25.121+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:25.242 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:25.237+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:25.244 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:25.240+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:25.562 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:25.556+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:25.755 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:25.752+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:25.977 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:25.971+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:25.994 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:25.989+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:26.016 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:26.012+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:26.141 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:26.136+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:26.214 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:26.209+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:26.216 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:26.213+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:26.547 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:26.542+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:26.787 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:26.782+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:26.967 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:26.962+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:26.979 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:26.974+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:26.990 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:26.986+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:27.166 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:27.161+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:27.225 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:27.220+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:27.227 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:27.223+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:27.540 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:27.534+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:27.826 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:27.822+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:27.952 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:27.947+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:27.968 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:27.963+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:27.996 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:27.991+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:28.212 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:28.208+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:28.217 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:28.211+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:28.218 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:28.213+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:28.524 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:28.520+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:28.852 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:28.848+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:28.952 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:28.949+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:28.990 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:28.984+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:29.028 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:29.025+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:29.174 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:29.168+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:29.228 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:29.222+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:29.229 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:29.223+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:29.503 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:29.497+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:29.830 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:29.826+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:29.969 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:29.965+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:30.027 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:30.022+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:30.053 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:30.050+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:30.143 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:30.138+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:30.239 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:30.233+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:30.239 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:30.235+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:30.548 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:30.543+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:30.878 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:30.873+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:30.947 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:30.944+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:31.000 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:30.994+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:31.015 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:31.012+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:31.166 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:31.160+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:31.214 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:31.207+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:31.214 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:31.209+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:31.561 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:31.557+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:31.860 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:31.857+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:31.958 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:31.953+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:31.989 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:31.985+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:32.045 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:32.041+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:32.122 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:32.117+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:32.261 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:32.255+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:32.271 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:32.259+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:32.589 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:32.583+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:32.878 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:32.875+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:32.948 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:32.943+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:33.043 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:33.040+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:33.072 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:33.068+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:33.120 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:33.115+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:33.258 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:33.252+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:33.259 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:33.253+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:33.617 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:33.611+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:33.924 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:33.922+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:33.986 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:33.981+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:34.016 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:34.012+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:34.060 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:34.057+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:34.154 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:34.148+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:34.261 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:34.255+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:34.261 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:34.256+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:34.599 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:34.593+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:34.936 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:34.933+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:35.034 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:35.031+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:35.034 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:35.030+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:35.039 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:35.036+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:35.182 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:25:56.642 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.187 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:35.182+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:35.193 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:25:56.653 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.203 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:25:56.686 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.205 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:25:56.665 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.206 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:25:56.689 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.212 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:25:56.671 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:35.299+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:35.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:35.300+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:35.449 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:25:56.931 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.484 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:25:56.944 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.495 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:25:56.955 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:54:35.629 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:35.624+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:35.919 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:35.915+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:35.997 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:35.992+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:36.072 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:36.065+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:36.093 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:36.088+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:36.189 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:36.181+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:36.333 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:36.328+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:36.335 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:36.331+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:36.659 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:36.655+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:36.944 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:36.940+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:36.988 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:36.983+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:37.074 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:37.070+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:37.103 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:37.099+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:37.143 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:37.139+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:37.323 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:37.317+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:37.324 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:37.319+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:37.646 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:37.641+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:37.936 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:37.932+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:37.967 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:37.961+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:38.059 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:38.056+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:38.135 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:38.131+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:38.143 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:38.138+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:38.298 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:38.292+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:38.298 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:38.293+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:38.674 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:38.668+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:38.965 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:38.961+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:38.989 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:38.985+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:39.072 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:39.067+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:39.153 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:39.148+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:39.172 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:39.168+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:39.322 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:39.317+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:39.323 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:39.320+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:39.662 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:39.656+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:39.954 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:39.948+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:39.976 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:39.972+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:40.031 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:40.026+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:40.144 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:40.138+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:40.218 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:40.213+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:40.349 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:40.344+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:40.352 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:40.348+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:40.683 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:40.678+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:40.971 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:40.966+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:40.989 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:40.985+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:41.019 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:41.015+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:41.152 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:41.147+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:41.230 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:41.225+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:41.347 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:41.342+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:41.350 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:41.345+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:41.657 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:41.650+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:41.986 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:41.981+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:41.987 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:41.983+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:42.011 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:42.006+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:42.137 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:42.132+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:42.230 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:42.225+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:42.398 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:42.392+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:42.398 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:42.393+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:42.697 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:42.690+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:42.946 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:42.942+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:42.967 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:42.962+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:42.988 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:42.983+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:43.100 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:43.095+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:43.204 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:43.200+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:43.392 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:43.387+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:43.393 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:43.388+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:43.651 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:43.646+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:43.988 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:43.984+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:44.002 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:43.997+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:44.017 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:44.012+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:44.077 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:44.071+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:44.227 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:44.223+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:44.393 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:44.380+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:44.397 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:44.382+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:44.620 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:44.615+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:44.953 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:44.949+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:44.964 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:44.959+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:45.059 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:45.055+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:45.062 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:45.057+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:45.217 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:45.212+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:45.429 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:45.424+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:45.431 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:45.427+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:45.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:45.660+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:45.934 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:45.930+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:46.023 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:46.018+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:46.025 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:46.005+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:46.103 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:46.098+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:46.252 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:46.248+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:46.423 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:46.418+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:46.426 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:46.422+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:46.668 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:46.659+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:46.906 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:46.902+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:46.967 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:46.962+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:46.997 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:46.991+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:47.056 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:47.051+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:47.244 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:47.236+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:47.384 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:47.377+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:47.385 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:47.380+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:47.711 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:47.705+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:47.925 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:47.920+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:47.995 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:47.991+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:48.008 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:48.001+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:48.019 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:54:48.011+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 41sec e212 v212) 2019-12-09T14:54:48.065 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:48.059+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:48.242 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:48.237+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:48.354 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:48.348+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:48.355 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:48.349+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:48.734 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:48.726+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:48.884 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:48.879+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:48.973 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:48.969+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:48.989 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:48.983+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:49.062 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:49.056+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:49.265 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:49.261+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:49.372 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:49.366+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:49.374 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:49.369+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:49.763 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:49.756+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:49.925 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:49.920+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:49.930 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:49.925+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:49.980 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:49.974+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:50.098 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:50.091+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:50.251 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:50.247+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:50.347 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:50.341+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:50.348 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:50.342+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:50.795 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:50.789+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:50.889 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:50.885+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:50.959 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:50.954+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:51.024 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:51.019+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:51.117 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:51.111+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:51.301 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:51.296+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:51.308 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:51.300+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:51.308 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:51.302+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:51.766 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:51.760+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:51.939 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:51.932+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:51.964 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:51.960+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:52.081 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:52.073+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:52.126 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:52.118+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:52.302 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:52.297+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:52.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:52.318+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:52.327 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:52.322+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:52.734 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:52.728+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:52.902 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:52.898+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:52.954 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:52.950+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:53.086 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:54:53.077+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 41sec e212 v212) 2019-12-09T14:54:53.101 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:53.095+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:53.107 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:53.101+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:53.304 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:53.300+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:53.315 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:53.309+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:53.316 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:53.310+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:53.709 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:53.702+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:53.932 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:53.928+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:53.949 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:53.946+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:54.102 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:54.096+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:54.111 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:54.104+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:54.329 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:54.325+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:54.344 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:54.337+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:54.347 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:54.341+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:54.760 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:54.754+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:54.951 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:54.947+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:54.967 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:54.963+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:55.138 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:55.132+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:55.142 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:55.135+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:55.343 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:55.338+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:55.354 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:55.347+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:55.354 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:55.348+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:55.758 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:55.752+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:55.940 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:55.936+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:55.962 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:55.959+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:56.158 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:56.144+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:56.158 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:56.152+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:56.356 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:56.351+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:56.367 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:56.359+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:56.368 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:56.361+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:56.776 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:56.769+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:56.988 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:56.984+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:56.991 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:56.985+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:57.133 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:57.127+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:57.200 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:57.194+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:57.357 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:54:57.352+0000 1694f700 -1 osd.4 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501706+0000 front 2019-12-09T14:53:36.447850+0000 (oldest deadline 2019-12-09T14:54:18.504011+0000) 2019-12-09T14:54:57.405 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:57.397+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:57.415 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:57.407+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:57.794 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:54:57.788+0000 1694f700 -1 osd.1 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.914083+0000 front 2019-12-09T14:53:36.194100+0000 (oldest deadline 2019-12-09T14:54:20.373129+0000) 2019-12-09T14:54:57.957 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:54:57.953+0000 1694f700 -1 osd.6 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.415277+0000 front 2019-12-09T14:53:35.226323+0000 (oldest deadline 2019-12-09T14:54:18.042457+0000) 2019-12-09T14:54:57.974 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:54:57.970+0000 1694f700 -1 osd.7 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.838233+0000 front 2019-12-09T14:53:35.934401+0000 (oldest deadline 2019-12-09T14:54:17.705258+0000) 2019-12-09T14:54:58.098 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:54:58.090+0000 1694f700 -1 osd.3 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:36.501660+0000 front 2019-12-09T14:53:36.712638+0000 (oldest deadline 2019-12-09T14:54:18.071850+0000) 2019-12-09T14:54:58.164 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:54:58.157+0000 1694f700 -1 osd.2 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:35.191476+0000 front 2019-12-09T14:53:34.092754+0000 (oldest deadline 2019-12-09T14:54:19.153871+0000) 2019-12-09T14:54:58.170 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:54:58.157+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 41sec e212 v212) 2019-12-09T14:54:58.427 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:58.420+0000 1694f700 -1 osd.0 212 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:53:37.242128+0000 front 2019-12-09T14:53:35.190817+0000 (oldest deadline 2019-12-09T14:54:16.560650+0000) 2019-12-09T14:54:58.428 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:58.421+0000 1694f700 -1 osd.0 212 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:54:59.433 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:54:59.427+0000 1694f700 -1 osd.0 213 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105148 3.3b 3:dc8bfaf7:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.652:head [delete] snapc 0=[] ondisk+write+known_if_redirected e212) 2019-12-09T14:55:41.468 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:55:41.344+0000 36f91700 -1 osd.5 218 build_incremental_map_msg missing incremental map 218 2019-12-09T14:55:43.427 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:27:04.886 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.447 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:27:04.907 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.463 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:27:04.923 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.469 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:27:04.951 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.473 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:27:04.955 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.489 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:27:04.949 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.533 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:27:04.992 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.553 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:27:05.035 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:43.587 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:27:05.047 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:55:57.527 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:55:57.520+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:55:57.895 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:55:57.889+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:55:57.896 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:55:57.892+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:55:58.261 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:55:58.255+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:55:58.278 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:55:58.271+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:55:58.530 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:55:58.523+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:55:58.924 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:55:58.918+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:55:58.925 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:55:58.921+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:55:59.064 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:55:59.057+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:55:59.271 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:55:59.266+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:55:59.325 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:55:59.319+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:55:59.578 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:55:59.572+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:55:59.927 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:55:59.921+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:55:59.928 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:55:59.924+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:00.080 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:00.072+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:00.264 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:00.258+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:00.312 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:00.306+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:00.529 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:00.524+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:00.628 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:00.622+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:00.899 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:00.892+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:00.899 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:00.895+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:01.073 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:01.065+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:01.254 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:01.248+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:01.288 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:01.282+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:01.487 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:01.482+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:01.628 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:01.621+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:01.936 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:01.929+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:01.936 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:01.932+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:02.062 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:02.055+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:02.243 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:02.238+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:02.295 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:02.289+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:02.526 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:02.520+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:02.600 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:02.594+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:02.908 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:02.903+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:02.909 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:02.905+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:03.054 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:03.046+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:03.288 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:03.282+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:03.303 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:03.297+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:03.525 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:03.519+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:03.559 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:03.554+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:03.949 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:03.944+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:03.949 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:03.945+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:04.061 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:04.055+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:04.279 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:04.271+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:04.284 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:04.279+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:04.496 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:04.488+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:04.523 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:04.518+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:04.919 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:04.913+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:04.920 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:04.916+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:05.069 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:05.064+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:05.272 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:05.267+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:05.311 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:05.307+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:05.501 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:05.489+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:05.518 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:05.513+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:05.937 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:05.932+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:05.938 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:05.934+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:06.028 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:06.023+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:06.230 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:06.225+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:06.353 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:06.346+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:06.476 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:06.470+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:06.561 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:06.555+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:06.945 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:06.939+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:06.946 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:06.942+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:07.024 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:07.018+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:07.191 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:07.186+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:07.335 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:07.329+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:07.506 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:07.501+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:07.552 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:07.546+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:07.983 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:07.977+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:07.984 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:07.980+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:08.016 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:08.011+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:08.200 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:08.194+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:08.374 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:08.370+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:08.375 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:08.371+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:08.561 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:08.556+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:08.579 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:08.574+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:08.982 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:08.975+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:08.983 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:08.977+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:09.033 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:09.028+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:09.238 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:09.232+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:09.334 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:09.328+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:09.410 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:09.406+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:09.565 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:09.561+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:09.607 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:09.602+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:10.006 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:10.002+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:10.010 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:10.006+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:10.033 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:10.027+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:10.272 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:10.267+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:10.356 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:10.350+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:10.369 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:10.366+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:10.591 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:10.585+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:10.598 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:10.594+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:10.998 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:10.993+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:11.061 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:11.055+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:11.062 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:11.058+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:11.313 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:11.308+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:11.342 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:11.336+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:11.359 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:11.356+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:11.577 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:11.572+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:11.642 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:11.637+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:12.022 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:12.016+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:12.075 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:12.069+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:12.076 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:12.072+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:12.312 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:12.306+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:12.347 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:12.343+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:12.360 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:12.354+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:12.554 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:12.549+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:12.683 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:12.678+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:12.987 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:12.982+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:13.047 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:13.042+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:13.047 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:13.044+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:13.332 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:13.327+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:13.347 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:13.338+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:13.357 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:13.351+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:13.594 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:13.590+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:13.715 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:13.708+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:13.977 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:13.970+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:14.022 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:14.016+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:14.022 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:14.018+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:14.322 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:14.317+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:14.360 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:14.356+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:14.365 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:14.360+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:14.581 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:14.576+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:14.737 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:14.731+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:15.006 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:15.000+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:15.016 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:15.011+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:15.017 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:15.013+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:15.314 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:15.304+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:15.315 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:15.312+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:15.369 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:15.364+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:15.561 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:15.556+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:15.785 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:15.779+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:16.023 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:16.017+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:16.023 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:16.020+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:16.038 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:16.032+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:16.320 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:16.314+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:16.322 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:16.315+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:16.353 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:16.348+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:16.535 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:16.531+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:16.739 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:16.733+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:16.987 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:16.982+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:16.988 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:16.985+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:17.015 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:17.010+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:17.312 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:17.308+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:17.327 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:17.322+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:17.343 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:17.338+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:17.542 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:17.538+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:17.763 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:17.756+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:17.982 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:17.978+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:17.985 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:17.983+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:18.047 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:18.043+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:18.272 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:18.269+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:18.291 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:18.284+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:18.327 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:18.324+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:18.545 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:18.541+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:18.793 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:18.787+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:19.018 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:19.013+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:19.018 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:19.015+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:19.020 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:19.015+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:19.257 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:19.254+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:19.326 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:19.320+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:19.383 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:19.379+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:19.514 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:19.509+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:19.758 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:19.750+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:20.005 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:19.998+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:20.007 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:20.002+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:20.008 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:20.005+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:20.228 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:20.225+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:20.345 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:20.340+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:20.384 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:20.380+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:20.559 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:20.555+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:20.725 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:20.720+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:21.034 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:21.028+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:21.061 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:21.057+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:21.064 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:21.062+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:21.245 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:21.243+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:21.365 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:21.357+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:21.415 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:21.411+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:21.560 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:21.555+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:21.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:21.738+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:22.038 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:22.032+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:22.045 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:22.040+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:22.046 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:22.043+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:22.289 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:22.285+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:22.396 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:22.391+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:22.405 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:22.399+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:22.602 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:22.598+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:22.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:22.740+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:23.039 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:23.034+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:23.040 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:23.036+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:23.068 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:23.061+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:23.285 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:23.277+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:23.392 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:23.387+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:23.410 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:23.405+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:23.564 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:23.559+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:23.733 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:23.727+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:24.087 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:24.082+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:24.088 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:24.084+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:24.108 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:24.103+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:24.338 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:24.331+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:24.358 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:24.353+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:24.437 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:24.431+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:24.526 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:24.522+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:24.754 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:24.749+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:25.115 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:25.108+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:25.133 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:25.128+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:25.134 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:25.130+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:25.292 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:25.288+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:25.319 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:25.312+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:25.436 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:25.430+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:25.572 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:25.567+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:25.722 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:25.715+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:26.120 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:26.115+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:26.155 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:26.151+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:26.159 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:26.156+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:26.280 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:26.272+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:26.300 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:26.294+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:26.404 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:26.397+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:26.619 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:26.614+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:26.726 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:26.720+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:27.172 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:27.166+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:27.182 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:27.176+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:27.182 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:27.178+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:27.305 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:27.300+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:27.322 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:27.317+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:27.372 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:27.366+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:27.654 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:27.649+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:27.697 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:27.691+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:28.175 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:28.168+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:28.189 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:28.185+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:28.192 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:28.189+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:28.326 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:28.322+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:28.344 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:28.339+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:28.373 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:28.367+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:28.688 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:28.681+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:28.734 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:28.729+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:29.150 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:29.145+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:29.212 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:29.207+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:29.215 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:29.212+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:29.329 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:29.326+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:29.393 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:29.388+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:29.396 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:29.390+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:29.693 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:29.688+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:29.728 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:29.719+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:30.200 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:30.193+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:30.202 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:30.196+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:30.203 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:30.199+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:30.292 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:30.288+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:30.367 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:30.361+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:30.412 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:30.406+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:30.730 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:30.724+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:30.759 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:30.753+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:31.187 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:31.182+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:31.216 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:31.211+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:31.219 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:31.216+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:31.325 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:31.321+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:31.390 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:31.384+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:31.414 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:31.409+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:31.740 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:31.735+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:31.742 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:31.737+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:32.185 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:32.179+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:32.274 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:32.264+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:32.276 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:32.266+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:32.364 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:32.359+0000 1694f700 -1 osd.5 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:55:34.494756+0000 (oldest deadline 2019-12-09T14:56:31.526662+0000) 2019-12-09T14:56:32.369 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:32.365+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:32.440 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:32.434+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:32.446 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:32.441+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:32.712 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:32.706+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:32.771 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:32.766+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:33.235 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:33.230+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:33.237 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:33.233+0000 1694f700 -1 osd.4 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.324766+0000 front 2019-12-09T14:55:12.951211+0000 (oldest deadline 2019-12-09T14:55:57.622926+0000) 2019-12-09T14:56:33.241 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:56:33.238+0000 1694f700 -1 osd.4 218 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105356 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e214) 2019-12-09T14:56:33.351 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:33.346+0000 1694f700 -1 osd.5 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:55:34.494756+0000 (oldest deadline 2019-12-09T14:56:31.526662+0000) 2019-12-09T14:56:33.355 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:33.351+0000 1694f700 -1 osd.5 218 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:33.419 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:56:33.414+0000 1694f700 -1 osd.6 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:14.610134+0000 front 2019-12-09T14:55:14.985062+0000 (oldest deadline 2019-12-09T14:55:57.460447+0000) 2019-12-09T14:56:33.443 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:56:33.438+0000 1694f700 -1 osd.3 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:12.817981+0000 front 2019-12-09T14:55:13.051720+0000 (oldest deadline 2019-12-09T14:55:57.439514+0000) 2019-12-09T14:56:33.565 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:56:33.551+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 43sec e218 v218) 2019-12-09T14:56:33.714 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:56:33.701+0000 1694f700 -1 osd.1 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.073828+0000 front 2019-12-09T14:55:14.840175+0000 (oldest deadline 2019-12-09T14:55:56.876481+0000) 2019-12-09T14:56:33.755 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:56:33.752+0000 1694f700 -1 osd.7 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:15.748692+0000 front 2019-12-09T14:55:15.751413+0000 (oldest deadline 2019-12-09T14:56:00.428901+0000) 2019-12-09T14:56:34.249 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:56:34.244+0000 1694f700 -1 osd.0 218 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:55:13.309484+0000 front 2019-12-09T14:55:13.322298+0000 (oldest deadline 2019-12-09T14:55:58.556749+0000) 2019-12-09T14:56:34.447 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:34.441+0000 1694f700 -1 osd.5 219 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e216) 2019-12-09T14:56:34.844 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T14:56:34.775+0000 32788700 -1 osd.5 219 build_incremental_map_msg missing incremental map 218 2019-12-09T14:57:04.375 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:57:04.371+0000 1694f700 -1 osd.4 221 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105467 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 821426992 gen 72] snapc 0=[] ondisk+write+known_if_redirected e219) 2019-12-09T14:58:14.949 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:14.944+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:14.950 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:14.947+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:15.907 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:15.903+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:15.935 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:15.930+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:16.899 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:16.892+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:16.981 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:16.975+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:17.414 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:17.409+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:17.463 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:17.458+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:17.895 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:17.891+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:17.968 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:17.964+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:18.026 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:18.020+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:18.406 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:18.399+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:18.479 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:18.473+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:18.681 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:18.676+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:18.944 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:18.939+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:18.981 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:18.966+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:19.052 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:19.047+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:19.116 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:19.110+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:19.447 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:19.441+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:19.533 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:19.527+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:19.666 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:19.662+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:19.961 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:19.957+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:19.980 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:19.976+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:20.077 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:20.072+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:20.113 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:20.108+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:20.456 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:20.450+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:20.519 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:20.512+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:20.717 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:20.713+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:20.925 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:20.920+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:20.948 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:20.945+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:21.053 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:21.048+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:21.118 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:21.113+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:21.413 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:21.408+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:21.512 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:21.505+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:21.711 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:21.707+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:21.920 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:21.915+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:21.978 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:21.973+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:22.086 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:22.078+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:22.165 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:22.159+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:22.400 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:22.394+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:22.553 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:22.542+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:22.756 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:22.751+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:22.973 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:22.966+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:23.009 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:23.004+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:23.101 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:23.095+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:23.158 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:23.152+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:23.364 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:23.357+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:23.512 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:23.506+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:23.771 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:23.767+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:23.966 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:23.962+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:24.050 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:24.045+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:24.072 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:24.067+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:24.207 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:24.201+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:24.356 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:24.350+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:24.484 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:24.479+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:24.778 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:24.773+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:24.930 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:24.925+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:25.027 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:25.023+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:25.101 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:25.097+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:25.174 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:25.168+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:25.315 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:25.309+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:25.515 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:25.509+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:25.796 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:25.790+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:25.961 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:25.953+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:26.070 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:26.066+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:26.130 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:29:47.590 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.135 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:29:47.595 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.144 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:26.137+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:26.155 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:29:47.615 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.161 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:29:47.643 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.164 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:29:47.624 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.164 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:26.159+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:26.169 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:29:47.651 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.320 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:26.314+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:26.336 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:29:47.818 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.419 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:29:47.878 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.465 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:29:47.925 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:26.481 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:26.476+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:26.753 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:26.748+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:26.952 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:26.947+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:27.115 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:27.111+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:27.121 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:27.116+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:27.158 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:27.152+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:27.308 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:27.303+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:27.475 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:27.469+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:27.712 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:27.708+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:27.928 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:27.923+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:28.135 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:28.129+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:28.136 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:28.132+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:28.170 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:28.165+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:28.288 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:28.282+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:28.524 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:28.518+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:28.737 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:28.733+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:28.960 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:28.956+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:29.126 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:29.121+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:29.146 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:29.140+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:29.177 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:29.171+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:29.252 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:29.246+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:29.561 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:29.556+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:29.743 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:29.738+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:30.004 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:30.000+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:30.146 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:30.141+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:30.155 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:30.150+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:30.180 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:30.176+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:30.223 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:30.217+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:30.553 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:30.548+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:30.786 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:30.782+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:31.020 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:31.015+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:31.122 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:31.117+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:31.181 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:31.177+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:31.194 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:31.189+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:31.206 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:31.201+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:31.579 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:31.573+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:31.746 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:31.742+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:31.988 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:31.983+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:32.144 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:32.138+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:32.168 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:32.163+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:32.215 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:32.209+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:32.220 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:32.213+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:32.537 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:32.532+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:32.743 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:32.738+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:32.967 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:32.962+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:33.148 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:33.144+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:33.187 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:33.181+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:33.189 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:33.184+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:33.189 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:33.185+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:33.515 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:33.509+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:33.766 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:33.762+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:33.993 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:33.989+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:34.128 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:34.124+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:34.183 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:34.175+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:34.192 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:34.186+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:34.200 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:34.195+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:34.562 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:34.557+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:34.783 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:34.779+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:34.965 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:34.961+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:35.147 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:35.144+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:35.210 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:35.204+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:35.225 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:35.219+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:35.253 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:35.249+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:35.567 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:35.558+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:35.825 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:35.822+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:35.978 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:35.970+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:36.175 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:36.170+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:36.186 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:36.180+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:36.238 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:36.232+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:36.274 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:36.268+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:36.576 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:36.571+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:36.826 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:36.823+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:36.997 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:36.992+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:37.139 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:37.133+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:37.182 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:37.177+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:37.276 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:37.272+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:37.289 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:37.284+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:37.596 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:37.590+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:37.789 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:37.784+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:38.002 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:37.997+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:38.154 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:38.148+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:38.172 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:38.164+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:38.272 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:38.267+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:38.315 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:38.308+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:38.634 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:38.629+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:38.828 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:38.824+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:39.018 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:39.014+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:39.152 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:39.148+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:39.200 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:39.195+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:39.298 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:39.292+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:39.323 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:39.317+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:39.628 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:39.622+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:39.863 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:39.858+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:40.005 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:40.001+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:40.120 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:40.116+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:40.245 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:40.239+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:40.257 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:40.251+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:40.288 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:40.282+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:40.652 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:40.646+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:40.831 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:40.826+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:41.022 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:41.017+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:41.094 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:41.089+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:41.240 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:41.235+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:41.262 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:41.257+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:41.343 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:41.337+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:41.665 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:41.659+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:41.856 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:41.852+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:42.052 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:42.047+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:42.102 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:42.097+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:42.203 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:42.197+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:42.246 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:42.240+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:42.392 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:42.386+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:42.649 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:42.643+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:42.860 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:42.856+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:43.013 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:43.007+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:43.098 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:43.093+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:43.184 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:43.179+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:43.268 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:43.263+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:43.446 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:43.440+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:43.618 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:43.612+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:43.865 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:43.861+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:43.970 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:43.965+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:44.080 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:44.075+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:44.219 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:44.213+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:44.268 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:44.263+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:44.444 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:44.437+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:44.633 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:44.625+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:44.864 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:44.859+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:44.971 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:44.967+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:45.127 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:45.119+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:45.215 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:45.211+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:45.294 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:45.288+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:45.492 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:45.486+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:45.656 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:45.651+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:45.838 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:45.834+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:45.931 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:45.927+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:46.160 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:46.154+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:46.235 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:46.228+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:46.259 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:46.252+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:46.477 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:46.470+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:46.661 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:46.654+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:46.878 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:46.873+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:46.904 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:46.900+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:47.166 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:47.161+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:47.201 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:47.193+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:47.231 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:47.225+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:47.493 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:47.486+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:47.661 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:47.655+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:47.908 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:47.904+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:47.936 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:47.932+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:48.138 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:48.134+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:48.190 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:48.179+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:48.270 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:48.263+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:48.521 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:48.513+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:48.693 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:48.686+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:48.886 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:48.881+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:48.946 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:58:48.939+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 47sec e229 v229) 2019-12-09T14:58:48.949 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:48.944+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:49.186 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:49.182+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:49.188 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:49.181+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:49.263 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:49.255+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:49.562 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:49.556+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:49.655 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:49.648+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:49.934 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:49.929+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:49.963 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:49.958+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:50.141 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:50.137+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:50.188 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:50.182+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:50.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:50.297+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:50.542 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:50.536+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:50.629 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:50.623+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:50.955 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:50.951+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:50.988 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:50.984+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:51.117 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:51.113+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:51.155 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:51.149+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:51.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:51.320+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:51.542 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:51.535+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:51.679 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:51.672+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:51.999 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:51.995+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:52.013 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:52.010+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:52.149 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:52.142+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:52.172 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:52.168+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:52.175 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:52.172+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:52.327 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:52.320+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:52.576 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:52.569+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:52.650 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:52.643+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:53.009 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:53.004+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:53.013 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:53.009+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:53.139 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:53.132+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:53.144 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:53.140+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:53.145 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:53.142+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:53.309 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:53.302+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:53.541 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:53.534+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:53.669 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:53.662+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:53.991 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:53.986+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:54.019 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:54.015+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:54.113 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:54.107+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:54.132 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:54.124+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:54.132 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:54.126+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:54.239 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:58:54.234+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 47sec e229 v229) 2019-12-09T14:58:54.282 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:30:15.741 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.311 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:54.305+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:54.318 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:30:15.778 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.362 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:30:15.822 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.378 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:30:15.838 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.390 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:30:15.872 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.392 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:30:15.875 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.399 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:30:15.859 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.427 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:30:15.909 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.432 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:30:15.891 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T14:58:54.588 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:54.582+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:54.679 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:54.673+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:55.015 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:55.011+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:55.043 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:55.039+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:55.093 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:55.087+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:55.180 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:55.176+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:55.180 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:55.177+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:55.345 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:55.338+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:55.635 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:55.630+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:55.651 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:55.645+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:56.068 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:56.064+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:56.078 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:56.074+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:56.109 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:56.099+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:56.212 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:56.207+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:56.213 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:56.209+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:56.318 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:56.311+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:56.594 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:56.587+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:56.648 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:56.642+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:57.051 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:57.047+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:57.064 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:57.059+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:57.113 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:57.106+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:57.214 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:57.210+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:57.215 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:57.211+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:57.317 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:57.311+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:57.608 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:57.599+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:57.642 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:57.635+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:58.062 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:58.058+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:58.110 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:58.106+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:58.111 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:58.103+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:58.245 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:58.242+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:58.246 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:58.243+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:58.284 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:58.278+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:58.588 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:58:58.581+0000 1694f700 -1 osd.3 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.955120+0000 front 2019-12-09T14:57:32.942038+0000 (oldest deadline 2019-12-09T14:58:16.750014+0000) 2019-12-09T14:58:58.655 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:58:58.649+0000 1694f700 -1 osd.1 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.277126+0000 front 2019-12-09T14:57:28.996171+0000 (oldest deadline 2019-12-09T14:58:14.766623+0000) 2019-12-09T14:58:59.081 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:58:59.078+0000 1694f700 -1 osd.7 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:34.938839+0000 front 2019-12-09T14:57:32.741506+0000 (oldest deadline 2019-12-09T14:58:18.306433+0000) 2019-12-09T14:58:59.100 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:58:59.097+0000 1694f700 -1 osd.6 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:29.280398+0000 front 2019-12-09T14:57:28.991544+0000 (oldest deadline 2019-12-09T14:58:14.693441+0000) 2019-12-09T14:58:59.151 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T14:58:59.145+0000 1694f700 -1 osd.2 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.383444+0000 front 2019-12-09T14:57:33.688652+0000 (oldest deadline 2019-12-09T14:58:16.706792+0000) 2019-12-09T14:58:59.208 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:59.204+0000 1694f700 -1 osd.4 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:32.036311+0000 front 2019-12-09T14:57:32.076071+0000 (oldest deadline 2019-12-09T14:58:17.147194+0000) 2019-12-09T14:58:59.209 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:58:59.206+0000 1694f700 -1 osd.4 229 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:58:59.283 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:58:59.277+0000 1694f700 -1 osd.0 229 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T14:57:33.344623+0000 front 2019-12-09T14:57:33.465240+0000 (oldest deadline 2019-12-09T14:58:18.446229+0000) 2019-12-09T14:58:59.302 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T14:58:59.294+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 47sec e229 v229) 2019-12-09T14:59:00.184 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:00.182+0000 1694f700 -1 osd.4 230 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:59:01.186 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:01.184+0000 1694f700 -1 osd.4 231 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105915 2.23s0 2:c5620bae:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650_fea67f8f551943b99634ba79d23df90f:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e229) 2019-12-09T14:59:32.367 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:32.365+0000 1694f700 -1 osd.4 231 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:33.411 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:33.408+0000 1694f700 -1 osd.4 231 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:34.417 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:34.414+0000 1694f700 -1 osd.4 231 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:35.462 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:35.459+0000 1694f700 -1 osd.4 232 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:36.461 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:36.453+0000 1694f700 -1 osd.4 232 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:37.448 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:37.445+0000 1694f700 -1 osd.4 233 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:38.479 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:38.476+0000 1694f700 -1 osd.4 234 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:39.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:39.491+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:40.543 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:40.540+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:41.584 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:41.581+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:42.629 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:42.626+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:43.640 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:43.637+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:44.687 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:44.683+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:45.675 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:45.671+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:46.637 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:46.632+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:47.667 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:47.664+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:48.644 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:48.640+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:49.602 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:49.598+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:50.573 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:50.569+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:51.532 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:51.530+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:52.573 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:52.570+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:53.592 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:53.589+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:54.574 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:54.572+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:55.626 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:55.623+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:56.167 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:59:56.160+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T14:59:56.637 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:56.635+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:57.127 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:59:57.121+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T14:59:57.615 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:57.612+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:57.644 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:59:57.640+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T14:59:57.979 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:59:57.973+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T14:59:58.065 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:59:58.057+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T14:59:58.178 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:59:58.171+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T14:59:58.320 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:59:58.316+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T14:59:58.616 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:58.610+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T14:59:58.616 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:58.613+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:58.677 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:59:58.671+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T14:59:58.962 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:59:58.956+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T14:59:59.078 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T14:59:59.070+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T14:59:59.220 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T14:59:59.212+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T14:59:59.322 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T14:59:59.316+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T14:59:59.640 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:59.634+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T14:59:59.642 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T14:59:59.637+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T14:59:59.645 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T14:59:59.636+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T14:59:59.964 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T14:59:59.957+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:00.086 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:00.080+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:00.186 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:00.179+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:00.350 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:00.345+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:00.644 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:00.639+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:00.656 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:00.651+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:00.657 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:00.653+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:01.003 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:00.997+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:01.060 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:01.051+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:01.153 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:01.146+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:01.404 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:01.398+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:01.685 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:01.679+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:01.689 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:01.684+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:01.691 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:01.684+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:02.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:02.004+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:02.093 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:02.086+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:02.124 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:02.118+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:02.440 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:02.434+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:02.668 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:02.663+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:02.672 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:02.668+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:02.727 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:02.722+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:03.032 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:03.026+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:03.095 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:03.089+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:03.134 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:03.127+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:03.477 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:03.472+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:03.686 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:03.680+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:03.687 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:03.683+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:03.703 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:03.698+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:04.077 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:04.068+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:04.097 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:04.086+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:04.147 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:04.141+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:04.528 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:04.522+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:04.652 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:04.648+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:04.653 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:04.650+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:04.675 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:04.671+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:05.104 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:05.099+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:05.112 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:05.106+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:05.179 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:05.174+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:05.572 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:05.567+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:05.630 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:05.625+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:05.634 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:05.631+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:05.703 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:05.697+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:06.102 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:06.097+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:06.138 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:06.133+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:06.150 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:06.145+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:06.532 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:06.527+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:06.667 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:06.662+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:06.667 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:06.661+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:06.671 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:06.667+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:07.117 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:07.112+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:07.161 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:07.155+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:07.170 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:07.166+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:07.576 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:07.571+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:07.668 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:07.663+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:07.702 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:07.695+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:07.703 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:07.698+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:08.134 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:08.125+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:08.137 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:08.132+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:08.163 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:08.158+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:08.533 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:08.528+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:08.694 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:08.688+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:08.695 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:08.689+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:08.695 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:08.691+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:09.137 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:09.132+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:09.145 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:09.140+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:09.154 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:09.149+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:09.174 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:09.169+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:09.507 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:09.502+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:09.653 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:09.647+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:09.654 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:09.650+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:09.673 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:09.667+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:10.101 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:10.097+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:10.144 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:10.138+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:10.158 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:10.153+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:10.172 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:10.166+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:10.485 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:10.481+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:10.667 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:10.662+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:10.685 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:10.680+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:10.686 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:10.682+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:11.085 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:11.080+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:11.099 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:11.094+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:11.145 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:11.140+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:11.189 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:11.184+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:11.483 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:11.479+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:11.678 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:11.672+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:11.697 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:11.691+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:11.698 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:11.694+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:12.083 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:12.078+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:12.093 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:12.088+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:12.140 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:12.132+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:12.213 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:12.208+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:12.468 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:12.461+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:12.675 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:12.668+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:12.676 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:12.672+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:12.729 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:12.724+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:13.119 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:13.114+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:13.138 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:13.133+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:13.180 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:13.175+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:13.196 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:13.191+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:13.436 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:13.429+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:13.683 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:13.677+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:13.684 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:13.680+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:13.739 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:13.735+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:14.101 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:14.096+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:14.110 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:14.104+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:14.228 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:14.222+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:14.245 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:14.239+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:14.468 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:14.463+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:14.645 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:14.640+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:14.647 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:14.644+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:14.764 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:14.757+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:15.106 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:15.100+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:15.108 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:15.103+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:15.157 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:15.152+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:15.217 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:15.212+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:15.238 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:15.233+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:15.459 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:15.455+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:15.632 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:15.627+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:15.645 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:15.632+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:15.735 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:15.729+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:16.101 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:16.095+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:16.102 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:16.098+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:16.179 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:16.171+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:16.219 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:16.213+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:16.280 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:16.272+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:16.499 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:16.494+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:16.664 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:16.658+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:16.665 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:16.661+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:16.708 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:16.702+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:17.154 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:17.148+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:17.155 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:17.151+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:17.194 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:17.189+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:17.213 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:17.203+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:17.286 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:17.279+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:17.535 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:17.530+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:17.705 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:17.699+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:17.706 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:17.702+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:17.722 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:17.717+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:18.133 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:18.127+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:18.137 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:18.134+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:18.218 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:18.209+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:18.237 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:18.229+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:18.254 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:18.247+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:18.500 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:18.494+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:18.665 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:18.659+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:18.666 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:18.662+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:18.739 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:18.734+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:19.170 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:19.165+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:19.174 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:19.170+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:19.208 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:19.202+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:19.258 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:19.253+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:19.275 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:19.269+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:19.524 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:19.520+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:19.643 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:19.637+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:19.644 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:19.639+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:19.792 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:19.788+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:20.158 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:20.153+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:20.162 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:20.159+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:20.193 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:20.187+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:20.235 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:20.230+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:20.294 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:20.289+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:20.521 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:20.516+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:20.670 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:20.665+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:20.673 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:20.667+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:20.752 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:20.748+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:21.162 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:21.155+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:21.205 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:21.199+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:21.206 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:21.202+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:21.282 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:21.277+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:21.292 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:21.286+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:21.482 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:21.477+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:21.665 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:21.659+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:21.666 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:21.662+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:21.720 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:21.714+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:22.150 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:22.144+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:22.160 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:22.156+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:22.164 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:22.160+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:22.245 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:22.239+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:22.297 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:22.292+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:22.493 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:22.487+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:22.624 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:22.618+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:22.632 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:22.621+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:22.684 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:22.679+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:23.149 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:23.143+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:23.150 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:23.146+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:23.173 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:23.166+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:23.252 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:23.247+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:23.273 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:23.267+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:23.511 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:23.507+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:23.582 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:23.578+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:23.585 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:23.582+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:23.700 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:23.695+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:24.145 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:24.139+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:24.149 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:24.145+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:24.159 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:24.153+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:24.222 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:24.215+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:24.308 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:24.301+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:24.494 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:24.488+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:24.581 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:24.576+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:24.583 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:24.580+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:24.738 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:24.733+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:25.146 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:25.141+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:25.150 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:25.146+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:25.196 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:25.189+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:25.269 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:25.262+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:25.283 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:25.277+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:25.545 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:25.540+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:25.580 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:25.574+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:25.581 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:25.577+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:25.732 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:25.727+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:26.178 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:26.172+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:26.179 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:26.174+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:26.197 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:26.190+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:26.259 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:26.254+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:26.279 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:26.273+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:26.566 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:26.562+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:26.626 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:26.622+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:26.630 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:26.627+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:26.735 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:26.730+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:27.136 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:27.130+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:27.137 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:27.132+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:27.243 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:27.238+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:27.268 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:27.264+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:27.286 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:27.281+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:27.607 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:27.601+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:27.664 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:27.658+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:27.664 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:27.660+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:27.794 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:27.787+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:28.109 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:28.103+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:28.110 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:28.106+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:28.248 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:28.237+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:28.257 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:28.251+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:28.307 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:28.300+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:28.601 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:28.596+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:28.711 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:28.706+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:28.715 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:28.712+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:28.764 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:28.760+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:29.114 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:29.109+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:29.117 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:29.114+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:29.251 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:29.246+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:29.263 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:29.258+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:29.314 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:29.309+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:29.524 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:00:29.518+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e235 v235) 2019-12-09T15:00:29.652 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:29.647+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:29.755 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:29.749+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:29.755 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:29.752+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:29.799 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:29.795+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:30.168 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:30.162+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:30.169 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:30.165+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:30.221 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:30.216+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:30.241 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:30.236+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:30.312 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:30.307+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:30.660 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:30.654+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:30.756 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:30.751+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:30.757 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:30.753+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:30.843 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:30.838+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:31.154 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:31.150+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:31.158 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:31.155+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:31.185 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:31.177+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:31.234 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:31.226+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:31.282 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:31.277+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:31.620 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:31.616+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:31.755 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:31.749+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:31.756 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:31.751+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:31.841 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:31.835+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:32.136 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:32.131+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:32.140 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:32.136+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:32.153 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:32.146+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:32.285 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:32.280+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:32.304 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:32.300+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:32.647 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:32.641+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:32.811 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:32.803+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:32.815 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:32.805+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:32.816 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:32.809+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:33.106 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:33.100+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:33.107 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:33.103+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:33.111 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:33.106+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:33.257 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:33.253+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:33.310 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:33.306+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:33.614 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:33.609+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:33.807 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:33.800+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:33.807 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:33.800+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:33.810 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:33.806+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:34.114 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:34.108+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:34.125 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:34.119+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:34.126 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:34.121+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:34.304 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:34.298+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:34.308 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:34.301+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:34.547 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:00:34.540+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e235 v235) 2019-12-09T15:00:34.627 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:34.621+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:34.795 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:34.790+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:34.823 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:34.818+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:34.824 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:34.820+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:35.091 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:35.085+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:35.092 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:35.087+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:35.112 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:35.105+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:35.304 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:35.295+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:35.336 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:35.330+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:35.637 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:35.632+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:35.792 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:35.786+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:35.811 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:35.806+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:35.814 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:35.811+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:36.091 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:36.082+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:36.139 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:36.133+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:36.140 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:36.136+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:36.342 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:36.336+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:36.347 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:36.342+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:36.645 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:36.641+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:36.774 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:36.769+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:36.862 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:36.859+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:36.866 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:36.863+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:37.088 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:37.082+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:37.108 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:37.103+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:37.109 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:37.105+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:37.322 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:37.316+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:37.331 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:37.326+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:37.655 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:37.651+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:37.737 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:37.732+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:37.877 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:37.872+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:37.880 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:37.877+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:38.076 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:38.070+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:38.078 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:38.071+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:38.080 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:38.075+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:38.303 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:38.297+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:38.310 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:38.303+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:38.649 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:38.644+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:38.735 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:38.729+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:38.900 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:38.895+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:38.901 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:38.897+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:39.086 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:39.081+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:39.089 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:39.086+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:39.127 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:39.121+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:39.281 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:39.275+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:39.354 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:39.348+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:39.559 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:00:39.552+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e235 v235) 2019-12-09T15:00:39.633 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:39.627+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:39.703 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:39.697+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:39.923 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:39.917+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:39.923 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:39.919+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:40.093 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:40.086+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:40.098 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:40.093+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:40.103 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:40.099+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:40.250 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:40.243+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:40.344 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:40.339+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:40.634 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:40.628+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:40.733 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:40.728+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:40.929 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:40.924+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:40.933 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:40.929+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:41.087 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:41.081+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:41.138 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:41.133+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:41.140 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:41.135+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:41.269 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:41.264+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:41.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:41.357+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:41.650 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:41.645+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:41.763 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:41.757+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:41.907 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:41.902+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:41.911 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:41.907+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:42.109 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:42.102+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:42.110 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:42.105+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:42.140 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:42.131+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:42.258 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:42.252+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:42.368 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:42.363+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:42.623 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:42.619+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:42.780 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:42.775+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:42.948 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:42.942+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:42.949 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:42.944+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:43.115 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:43.111+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:43.116 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:43.112+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:43.118 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:43.115+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:43.227 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:43.221+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:43.405 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:43.399+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:43.599 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:43.594+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:43.824 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:43.821+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:43.956 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:43.951+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:43.960 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:43.956+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:44.108 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:44.100+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:44.147 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:44.140+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:44.149 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:44.143+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:44.274 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:44.268+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:44.444 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:44.439+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:44.586 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:00:44.581+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e235 v235) 2019-12-09T15:00:44.614 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:44.609+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:44.803 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:44.799+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:44.924 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:44.918+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:44.925 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:44.921+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:45.103 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:45.094+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:45.191 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:45.187+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:45.193 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:45.189+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:45.279 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:45.273+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:45.428 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:45.422+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:45.664 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:45.660+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:45.765 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:45.762+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:45.943 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:45.939+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:45.944 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:45.941+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:46.067 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:46.062+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:46.209 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:46.204+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:46.211 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:46.207+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:46.273 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:46.267+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:46.436 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:46.430+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:46.676 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:46.672+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:46.804 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:46.800+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:46.904 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:46.899+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:46.905 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:46.902+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:47.081 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:47.074+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:47.232 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:47.227+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:47.233 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:47.229+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:47.281 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:47.276+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:47.432 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:47.427+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:47.705 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:47.701+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:47.841 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:47.837+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:47.865 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:47.861+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:47.866 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:47.863+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:48.108 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:48.101+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:48.229 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:48.224+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:48.230 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:48.227+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:48.278 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:48.272+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:48.462 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:48.456+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:48.696 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:48.692+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:48.817 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:48.813+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:48.898 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:48.893+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:48.899 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:48.896+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:49.096 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:49.087+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:49.233 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:49.228+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:49.234 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:49.231+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:49.237 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:49.230+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:49.455 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:49.448+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:49.599 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:00:49.592+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e235 v235) 2019-12-09T15:00:49.664 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:49.661+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:49.802 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:49.797+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:49.893 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:49.888+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:49.894 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:49.891+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:50.113 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:50.107+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:50.233 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:50.229+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:50.237 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:50.234+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:50.258 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:50.251+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:50.469 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:50.462+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:50.636 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:50.632+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:50.806 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:50.802+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:50.930 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:50.925+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:50.931 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:50.928+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:51.112 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:51.105+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:51.229 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:51.223+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:51.230 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:51.226+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:51.294 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:51.287+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:51.517 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:51.510+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:51.608 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:51.603+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:51.781 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:51.775+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:51.940 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:51.933+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:51.940 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:51.936+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:52.077 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:52.071+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:52.213 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:52.207+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:52.214 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:52.209+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:52.258 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:52.252+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:52.532 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:52.526+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:52.615 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:52.606+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:52.819 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:52.814+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:52.949 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:52.944+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:52.952 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:52.948+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:53.117 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:53.111+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:53.206 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:53.201+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:53.206 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:53.202+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:53.259 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:53.253+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:53.491 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:53.485+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:53.573 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:53.568+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:53.821 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:53.815+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:53.983 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:53.976+0000 1694f700 -1 osd.4 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.270991+0000 front 2019-12-09T14:59:11.844795+0000 (oldest deadline 2019-12-09T14:59:57.697334+0000) 2019-12-09T15:00:53.985 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:53.979+0000 1694f700 -1 osd.4 235 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:54.082 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:54.073+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:54.233 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:54.227+0000 1694f700 -1 osd.5 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T14:59:34.824799+0000 (oldest deadline 2019-12-09T15:00:14.824799+0000) 2019-12-09T15:00:54.234 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:00:54.230+0000 1694f700 -1 osd.5 235 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e233) 2019-12-09T15:00:54.235 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:54.227+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:54.535 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:00:54.527+0000 1694f700 -1 osd.3 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:13.085309+0000 front 2019-12-09T14:59:13.092693+0000 (oldest deadline 2019-12-09T14:59:55.948651+0000) 2019-12-09T15:00:54.602 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:00:54.597+0000 1694f700 -1 osd.7 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:12.756000+0000 front 2019-12-09T14:59:13.018702+0000 (oldest deadline 2019-12-09T14:59:57.330292+0000) 2019-12-09T15:00:54.649 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:00:54.632+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e235 v235) 2019-12-09T15:00:54.840 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:00:54.836+0000 1694f700 -1 osd.6 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.601963+0000 front 2019-12-09T14:59:14.631935+0000 (oldest deadline 2019-12-09T14:59:56.804425+0000) 2019-12-09T15:00:55.124 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:00:55.118+0000 1694f700 -1 osd.0 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:15.892362+0000 front 2019-12-09T14:59:15.961519+0000 (oldest deadline 2019-12-09T14:59:57.516993+0000) 2019-12-09T15:00:55.229 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:00:55.221+0000 1694f700 -1 osd.1 235 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T14:59:14.698511+0000 front 2019-12-09T14:59:14.729220+0000 (oldest deadline 2019-12-09T14:59:57.476013+0000) 2019-12-09T15:00:55.359 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:55.257+0000 1694f700 -1 osd.4 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:00:56.602 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:00:56.546+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:105983 3.2b 3:d4a27a91:::.dir.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.650:head [call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=160b] snapc 0=[] ondisk+write+known_if_redirected e231) 2019-12-09T15:02:15.529 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:15.525+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:15.673 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:15.666+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:16.131 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:16.127+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:16.554 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:16.550+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:16.700 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:16.693+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:16.711 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:16.703+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:16.737 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:16.731+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:17.169 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:17.165+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:17.601 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:17.596+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:17.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:17.662+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:17.750 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:17.744+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:17.758 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:17.752+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:17.896 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:17.891+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:18.180 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:18.176+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:18.580 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:18.574+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:18.698 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:18.692+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:18.707 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:18.702+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:18.718 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:18.712+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:18.733 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:18.727+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:18.915 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:18.910+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:19.157 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:19.151+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:19.531 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:33:40.991 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.537 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:33:40.996 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.551 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:19.547+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:19.554 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:33:41.013 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.558 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:33:41.017 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.562 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:33:41.044 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.574 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:33:41.056 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.660 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:19.654+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:19.676 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:19.671+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:19.714 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:19.709+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:19.715 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:19.711+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:19.784 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:33:41.243 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.802 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:33:41.261 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.893 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:33:41.375 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:02:19.953 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:19.947+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:20.176 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:20.167+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:20.563 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:20.559+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:20.642 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:20.637+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:20.668 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:20.663+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:20.683 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:20.678+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:20.707 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:20.701+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:20.905 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:20.902+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:21.215 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:21.211+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:21.578 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:21.574+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:21.644 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:21.639+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:21.655 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:21.650+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:21.691 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:21.687+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:21.720 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:21.716+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:21.870 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:21.863+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:22.176 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:22.172+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:22.609 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:22.604+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:22.657 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:22.652+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:22.705 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:22.699+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:22.720 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:22.714+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:22.758 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:22.753+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:22.833 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:22.829+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:23.203 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:23.199+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:23.621 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:23.616+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:23.628 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:23.622+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:23.739 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:23.733+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:23.749 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:23.744+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:23.782 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:23.776+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:23.866 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:23.861+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:24.197 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:24.192+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:24.628 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:24.624+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:24.633 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:24.626+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:24.711 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:24.705+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:24.763 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:24.757+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:24.774 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:24.768+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:24.919 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:24.914+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:25.224 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:25.219+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:25.603 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:25.597+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:25.633 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:25.628+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:25.673 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:25.667+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:25.794 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:25.789+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:25.797 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:25.792+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:25.923 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:25.917+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:26.206 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:26.202+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:26.583 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:26.576+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:26.647 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:26.643+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:26.707 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:26.702+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:26.805 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:26.800+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:26.823 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:26.818+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:26.954 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:26.949+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:27.227 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:27.223+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:27.565 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:27.560+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:27.609 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:27.604+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:27.702 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:27.696+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:27.796 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:27.791+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:27.823 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:27.817+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:27.934 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:27.929+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:28.267 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:28.263+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:28.604 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:28.595+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:28.606 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:28.600+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:28.729 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:28.723+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:28.785 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:28.778+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:28.814 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:28.808+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:28.942 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:28.936+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:29.271 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:29.267+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:29.618 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:29.611+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:29.632 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:29.626+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:29.729 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:29.724+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:29.834 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:29.828+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:29.858 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:29.853+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:29.914 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:29.909+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:30.237 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:30.233+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:30.579 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:30.575+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:30.648 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:30.642+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:30.780 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:30.774+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:30.858 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:30.853+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:30.877 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:30.872+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:30.948 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:30.943+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:31.284 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:31.279+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:31.579 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:31.574+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:31.653 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:31.647+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:31.833 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:31.827+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:31.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:31.873+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:31.883 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:31.878+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:31.965 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:31.961+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:32.322 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:32.317+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:32.616 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:32.612+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:32.675 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:32.669+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:32.796 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:32.789+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:32.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:32.835+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:32.938 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:32.932+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:32.959 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:32.957+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:33.357 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:33.354+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:33.624 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:33.620+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:33.645 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:33.640+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:33.849 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:33.843+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:33.863 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:33.857+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:33.927 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:33.923+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:33.931 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:33.925+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:34.387 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:34.384+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:34.627 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:34.623+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:34.697 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:34.692+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:34.858 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:34.852+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:34.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:34.858+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:34.892 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:34.887+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:34.895 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:34.891+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:35.369 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:35.366+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:35.670 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:35.666+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:35.673 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:35.667+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:35.846 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:35.841+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:35.849 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:35.844+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:35.881 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:35.877+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:35.913 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:35.905+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:36.338 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:36.334+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:36.674 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:36.667+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:36.684 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:36.680+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:36.846 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:36.843+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:36.856 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:36.852+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:36.890 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:36.883+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:36.964 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:36.957+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:37.375 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:37.372+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:37.656 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:37.652+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:37.711 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:37.705+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:37.800 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:37.797+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:37.839 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:37.835+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:37.927 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:37.922+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:37.951 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:37.946+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:38.378 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:38.374+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:38.666 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:38.662+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:38.719 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:38.713+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:38.843 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:38.837+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:38.853 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:38.849+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:38.942 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:38.936+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:38.953 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:38.945+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:39.387 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:39.382+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:39.686 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:39.680+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:39.728 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:39.722+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:39.809 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:39.804+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:39.887 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:39.881+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:39.908 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:39.902+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:39.959 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:39.953+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:40.399 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:40.394+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:40.671 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:40.667+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:40.772 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:40.767+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:40.776 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:40.770+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:40.898 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:40.892+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:40.950 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:40.945+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:40.982 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:40.977+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:41.367 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:41.362+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:41.717 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:41.713+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:41.733 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:41.729+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:41.745 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:41.739+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:41.887 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:41.880+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:41.952 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:41.946+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:41.983 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:41.978+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:42.370 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:42.365+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:42.737 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:42.732+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:42.753 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:42.748+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:42.772 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:42.765+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:42.875 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:42.868+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:42.921 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:42.915+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:43.009 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:43.003+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:43.357 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:43.354+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:43.709 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:43.704+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:43.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:43.738+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:43.779 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:43.774+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:43.865 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:43.858+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:43.933 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:43.927+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:44.005 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:43.998+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:44.325 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:44.320+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:44.698 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:44.695+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:44.749 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:44.743+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:44.809 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:44.804+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:44.870 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:44.864+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:44.933 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:44.927+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:44.979 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:44.974+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:45.302 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:45.297+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:45.735 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:45.730+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:45.782 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:45.777+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:45.804 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:45.799+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:45.897 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:45.892+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:45.941 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:45.935+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:45.970 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:45.966+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:46.353 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:46.349+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:46.721 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:02:46.716+0000 1694f700 -1 osd.7 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:30.403360+0000 front 2019-12-09T15:01:30.919968+0000 (oldest deadline 2019-12-09T15:02:14.876692+0000) 2019-12-09T15:02:46.819 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:46.813+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:46.844 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:46.839+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:46.904 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:46.897+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:46.927 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:46.922+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:47.021 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:47.016+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:47.388 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:47.384+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:47.781 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:47.775+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:47.804 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:47.800+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:47.934 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:47.929+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:47.938 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:47.933+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:48.036 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:48.031+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:48.426 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:48.421+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:48.747 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:48.741+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:48.844 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:48.840+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:48.919 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:48.913+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:48.965 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:48.959+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:49.052 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:49.048+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:49.429 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:49.424+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:49.786 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:49.781+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:49.884 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:49.878+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:49.898 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:49.890+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:49.970 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:49.964+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:50.083 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:50.077+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:50.162 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:02:50.155+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e240 v240) 2019-12-09T15:02:50.462 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:50.458+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:50.839 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:50.833+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:50.900 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:50.893+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:50.938 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:50.932+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:50.967 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:50.960+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:51.040 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:51.034+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:51.482 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:51.477+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:51.846 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:51.840+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:51.881 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:51.876+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:51.926 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:51.921+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:51.949 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:51.943+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:52.008 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:52.002+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:52.493 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:52.489+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:52.886 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:52.879+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:52.936 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:52.930+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:52.962 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:52.957+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:52.987 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:52.981+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:52.987 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:52.980+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:53.476 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:53.472+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:53.929 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:53.923+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:53.937 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:53.931+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:54.006 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:54.002+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:54.011 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:54.004+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:54.021 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:54.014+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:54.494 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:54.481+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:54.966 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:54.959+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:54.966 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:54.959+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:54.975 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:54.971+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:54.986 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:54.980+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:55.050 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:55.043+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:55.170 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:02:55.164+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e240 v240) 2019-12-09T15:02:55.444 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:55.439+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:55.946 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:55.940+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:55.955 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:55.948+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:55.959 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:55.955+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:56.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:56.005+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:56.016 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:56.010+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:56.490 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:56.485+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:56.949 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:56.945+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:56.967 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:56.961+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:56.972 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:56.962+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:57.014 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:57.004+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:57.017 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:57.010+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:57.451 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:57.446+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:57.910 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:57.906+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:57.929 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:57.921+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:57.999 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:57.992+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:58.002 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:57.994+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:58.047 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:58.040+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:58.442 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:58.437+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:58.904 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:58.899+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:58.944 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:58.937+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:02:59.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:02:59.005+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:02:59.031 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:02:59.024+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:02:59.080 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:02:59.072+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:02:59.401 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:02:59.397+0000 1694f700 -1 osd.6 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147756+0000 front 2019-12-09T15:01:32.541266+0000 (oldest deadline 2019-12-09T15:02:15.285091+0000) 2019-12-09T15:02:59.901 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:02:59.897+0000 1694f700 -1 osd.4 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:35.361286+0000 front 2019-12-09T15:01:35.207836+0000 (oldest deadline 2019-12-09T15:02:17.223697+0000) 2019-12-09T15:02:59.944 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:02:59.937+0000 1694f700 -1 osd.3 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.413332+0000 front 2019-12-09T15:01:29.283585+0000 (oldest deadline 2019-12-09T15:02:16.630015+0000) 2019-12-09T15:03:00.030 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:03:00.024+0000 1694f700 -1 osd.0 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:33.409641+0000 front 2019-12-09T15:01:35.295172+0000 (oldest deadline 2019-12-09T15:02:17.918818+0000) 2019-12-09T15:03:00.078 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:03:00.072+0000 1694f700 -1 osd.1 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:34.161031+0000 front 2019-12-09T15:01:33.417707+0000 (oldest deadline 2019-12-09T15:02:14.837423+0000) 2019-12-09T15:03:00.111 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:03:00.105+0000 1694f700 -1 osd.2 240 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:01:32.147169+0000 front 2019-12-09T15:01:33.321790+0000 (oldest deadline 2019-12-09T15:02:16.511808+0000) 2019-12-09T15:03:00.236 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:03:00.227+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e240 v240) 2019-12-09T15:03:57.140 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:03:57.134+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:03:58.069 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:03:58.063+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:03:58.131 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:03:58.125+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:03:58.814 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:03:58.807+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:03:59.049 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:03:59.042+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:03:59.164 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:03:59.159+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:03:59.345 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:03:59.339+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:03:59.511 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:03:59.504+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:03:59.801 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:03:59.793+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:00.039 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:00.033+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:00.123 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:00.118+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:00.328 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:00.320+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:00.490 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:00.485+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:00.814 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:00.808+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:00.986 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:00.977+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:00.998 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:00.993+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:01.137 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:01.130+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:01.308 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:01.301+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:01.492 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:01.487+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:01.808 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:01.802+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:01.954 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:01.948+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:02.046 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:02.040+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:02.155 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:02.149+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:02.338 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:02.330+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:02.465 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:02.457+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:02.836 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:02.826+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:02.932 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:02.924+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:03.035 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:03.030+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:03.194 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:03.189+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:03.361 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:03.354+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:03.443 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:03.438+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:03.803 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:03.795+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:03.904 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:03.897+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:03.991 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:03.986+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:04.212 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:04.206+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:04.329 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:04.321+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:04.446 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:04.440+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:04.772 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:04.765+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:04.927 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:04.920+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:05.022 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:05.017+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:05.217 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:05.212+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:05.294 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:05.288+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:05.458 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:05.453+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:05.566 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:05.561+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:05.805 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:05.799+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:05.945 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:05.938+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:06.065 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:06.060+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:06.212 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:06.208+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:06.346 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:06.338+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:06.414 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:06.409+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:06.612 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:06.591+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:06.836 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:06.830+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:06.995 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:06.987+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:07.092 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:07.086+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:07.214 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:07.208+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:07.324 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:07.318+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:07.393 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:07.387+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:07.569 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:07.565+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:07.801 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:07.794+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:07.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:07.974+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:08.137 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:08.129+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:08.194 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:08.189+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:08.352 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:08.346+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:08.440 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:08.434+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:08.559 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:08.554+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:08.791 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:08.785+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:09.032 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:09.026+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:09.187 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:09.183+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:09.204 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:09.199+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:09.313 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:09.307+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:09.442 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:09.438+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:09.596 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:09.590+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:09.836 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:09.827+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:09.996 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:09.990+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:10.219 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:10.214+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:10.231 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:10.226+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:10.301 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:10.295+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:10.426 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:10.420+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:10.647 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:10.642+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:10.835 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:10.828+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:10.967 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:10.961+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:11.191 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:11.186+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:11.221 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:11.209+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:11.310 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:11.305+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:11.457 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:11.452+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:11.618 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:11.614+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:11.837 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:11.831+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:12.012 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:12.005+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:12.192 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:12.187+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:12.208 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:12.203+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:12.333 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:12.326+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:12.436 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:12.430+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:12.639 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:12.635+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:12.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:12.855+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:12.996 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:12.991+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:13.167 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:13.162+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:13.177 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:13.173+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:13.297 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:13.283+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:13.465 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:13.460+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:13.672 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:13.667+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:13.676 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:13.672+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:13.867 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:13.862+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:13.975 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:13.970+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:14.213 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:14.208+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:14.225 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:14.220+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:14.285 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:14.279+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:14.500 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:14.494+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:14.638 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:14.633+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:14.642 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:14.638+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:14.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:14.876+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:15.011 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:15.006+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:15.231 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:15.226+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:15.245 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:15.239+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:15.254 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:15.249+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:15.537 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:15.533+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:15.685 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:15.678+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:15.687 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:15.683+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:15.923 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:15.916+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:16.021 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:16.015+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:16.192 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:16.186+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:16.263 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:16.258+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:16.295 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:16.288+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:16.516 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:16.510+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:16.704 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:16.699+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:16.708 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:16.705+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:16.939 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:16.933+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:17.029 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:17.023+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:17.189 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:17.183+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:17.280 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:17.273+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:17.290 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:17.284+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:17.475 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:17.469+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:17.679 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:17.674+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:17.680 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:17.676+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:17.980 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:17.974+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:18.013 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:18.006+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:18.226 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:18.221+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:18.265 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:18.256+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:18.298 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:18.291+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:18.488 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:18.484+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:18.653 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:18.647+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:18.657 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:18.653+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:18.977 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:18.970+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:18.986 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:18.979+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:19.193 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:19.188+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:19.305 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:19.298+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:19.338 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:19.332+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:19.483 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:19.478+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:19.700 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:19.695+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:19.704 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:19.700+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:19.999 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:19.993+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:20.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:20.005+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:20.183 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:20.179+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:20.338 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:20.333+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:20.385 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:20.377+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:20.470 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:20.465+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:20.733 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:20.728+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:20.733 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:20.730+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:20.988 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:20.982+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:21.045 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:21.039+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:21.143 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:21.138+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:21.311 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:21.307+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:21.435 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:21.430+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:21.507 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:21.502+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:21.695 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:21.690+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:21.696 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:21.692+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:22.029 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:22.024+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:22.099 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:22.092+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:22.138 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:22.133+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:22.314 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:22.310+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:22.447 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:22.441+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:22.542 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:22.537+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:22.674 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:22.669+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:22.677 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:22.675+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:23.039 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:23.033+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:23.100 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:23.096+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:23.109 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:23.103+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:23.273 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:23.269+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:23.408 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:23.402+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:23.544 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:23.539+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:23.639 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:23.635+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:23.643 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:23.639+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:24.014 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:24.008+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:24.099 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:24.095+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:24.128 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:24.123+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:24.253 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:24.249+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:24.406 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:24.399+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:24.586 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:24.581+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:24.646 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:24.642+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:24.652 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:24.647+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:25.039 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:25.034+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:25.120 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:25.115+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:25.171 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:25.165+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:25.212 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:25.207+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:25.435 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:25.430+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:25.582 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:25.577+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:25.634 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:25.629+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:25.635 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:25.632+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:25.994 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:25.989+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:26.127 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:26.123+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:26.158 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:26.153+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:26.259 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:26.255+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:26.392 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:26.386+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:26.570 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:26.564+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:26.680 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:26.673+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:26.681 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:26.676+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:26.980 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:26.975+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:27.150 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:27.144+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:27.178 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:27.173+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:27.307 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:27.301+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:27.429 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:27.424+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:27.528 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:27.521+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:27.708 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:27.703+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:27.712 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:27.708+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:27.949 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:27.943+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:28.163 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:28.158+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:28.178 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:28.171+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:28.275 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:28.270+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:28.471 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:28.464+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:28.549 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:28.544+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:28.673 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:28.667+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:28.674 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:28.670+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:28.961 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:28.955+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:29.185 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:29.180+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:29.204 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:29.199+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:29.296 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:29.289+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:29.473 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:29.468+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:29.545 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:29.539+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:29.690 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:29.684+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:29.691 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:29.687+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:30.011 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:30.004+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:30.151 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:30.145+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:30.188 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:30.182+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:30.347 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:30.341+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:30.508 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:30.502+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:30.562 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:04:30.557+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 46sec e246 v246) 2019-12-09T15:04:30.567 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:30.557+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:30.738 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:30.732+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:30.740 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:30.735+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:31.020 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:31.012+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:31.125 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:31.120+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:31.235 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:31.229+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:31.360 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:31.354+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:31.490 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:31.485+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:31.525 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:31.518+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:31.718 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:31.712+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:31.719 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:31.714+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:32.005 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:32.000+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:32.119 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:32.113+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:32.226 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:32.220+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:32.326 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:32.320+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:32.450 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:32.443+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:32.550 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:32.545+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:32.742 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:32.737+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:32.746 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:32.742+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:33.024 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:33.017+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:33.161 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:33.155+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:33.273 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:33.266+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:33.353 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:33.347+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:33.477 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:33.472+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:33.563 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:33.556+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:33.799 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:33.794+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:33.803 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:33.799+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:34.019 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:34.012+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:34.115 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:34.110+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:34.259 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:34.252+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:34.329 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:34.324+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:34.451 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:34.444+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:34.588 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:34.582+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:34.770 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:34.763+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:34.771 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:34.766+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:35.027 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:35.022+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:35.115 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:35.109+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:35.257 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:35.251+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:35.362 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:35.357+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:35.495 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:35.488+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:35.575 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:04:35.568+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 46sec e246 v246) 2019-12-09T15:04:35.619 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:35.614+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:35.764 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:35.759+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:35.765 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:35.760+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:36.045 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:36.040+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:36.159 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:36.154+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:36.267 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:36.260+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:36.325 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:36.320+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:36.486 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:36.480+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:36.617 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:36.611+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:36.799 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:36.793+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:36.800 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:36.795+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:37.010 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:37.004+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:37.158 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:37.153+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:37.309 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:37.303+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:37.379 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:37.374+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:37.511 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:37.506+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:37.600 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:37.594+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:37.802 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:37.797+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:37.806 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:37.802+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:38.029 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:38.022+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:38.201 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:38.196+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:38.312 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:38.304+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:38.391 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:38.386+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:38.541 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:38.536+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:38.608 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:38.602+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:38.773 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:38.768+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:38.778 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:38.773+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:39.063 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:39.056+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:39.240 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:39.234+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:39.265 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:39.259+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:39.358 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:39.353+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:39.582 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:39.577+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:39.641 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:39.636+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:39.734 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:39.728+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:39.735 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:39.730+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:40.028 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:40.021+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:40.249 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:40.243+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:40.252 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:40.247+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:40.332 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:40.325+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:40.595 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:04:40.589+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 46sec e246 v246) 2019-12-09T15:04:40.601 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:40.595+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:40.688 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:40.683+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:40.695 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:40.687+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:40.696 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:40.689+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:40.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:40.977+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:41.275 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:41.269+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:41.283 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:41.277+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:41.333 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:41.327+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:41.597 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:41.592+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:41.651 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:41.645+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:41.652 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:41.647+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:41.682 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:41.677+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:41.959 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:41.953+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:42.316 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:42.309+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:42.337 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:42.330+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:42.386 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:42.381+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:42.615 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:42.608+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:42.639 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:42.633+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:42.693 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:42.687+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:42.694 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:42.690+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:42.920 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:42.913+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:43.298 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:43.293+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:43.318 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:43.312+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:43.365 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:43.359+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:43.630 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:43.624+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:43.687 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:43.682+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:43.703 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:43.699+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:43.711 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:43.707+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:43.963 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:43.956+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:44.287 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:44.282+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:44.336 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:44.331+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:44.380 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:44.374+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:44.615 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:44.608+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:44.677 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:44.670+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:44.678 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:44.673+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:44.718 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:44.712+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:45.012 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:45.007+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:45.247 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:45.242+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:45.330 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:45.324+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:45.349 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:45.344+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:45.573 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:45.566+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:45.612 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:04:45.605+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 46sec e246 v246) 2019-12-09T15:04:45.667 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:45.661+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:45.668 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:45.663+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:45.696 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:45.690+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:46.013 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:46.008+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:46.229 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:46.223+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:46.314 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:46.308+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:46.324 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:46.318+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:46.549 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:46.543+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:46.637 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:46.631+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:46.638 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:46.633+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:46.654 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:46.649+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:47.007 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:47.001+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:47.206 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:47.201+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:47.271 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:47.266+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:47.309 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:47.303+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:47.532 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:47.527+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:47.539 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:47.533+0000 1694f700 -1 osd.1 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106677 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e246) 2019-12-09T15:04:47.656 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:47.650+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:47.657 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:47.653+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:47.687 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:47.681+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:47.973 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:47.968+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:48.243 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:48.238+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:48.256 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:48.251+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:48.272 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:48.267+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:48.564 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:48.558+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:48.566 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:48.561+0000 1694f700 -1 osd.1 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106677 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e246) 2019-12-09T15:04:48.662 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:48.657+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:48.694 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:48.689+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:48.697 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:48.694+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:49.009 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:49.002+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:49.267 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:49.262+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:49.268 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:49.263+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:49.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:49.298+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:49.552 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:49.547+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:49.553 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:49.548+0000 1694f700 -1 osd.1 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106677 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e246) 2019-12-09T15:04:49.666 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:49.661+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:49.706 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:49.702+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:49.708 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:49.705+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:49.992 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:04:49.986+0000 1694f700 -1 osd.3 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:16.707207+0000 front 2019-12-09T15:03:16.801991+0000 (oldest deadline 2019-12-09T15:04:00.790167+0000) 2019-12-09T15:04:50.286 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:04:50.281+0000 1694f700 -1 osd.6 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.370808+0000 front 2019-12-09T15:03:16.974256+0000 (oldest deadline 2019-12-09T15:03:56.848731+0000) 2019-12-09T15:04:50.294 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:04:50.289+0000 1694f700 -1 osd.7 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:12.486422+0000 front 2019-12-09T15:03:12.488529+0000 (oldest deadline 2019-12-09T15:03:57.474440+0000) 2019-12-09T15:04:50.319 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:04:50.312+0000 1694f700 -1 osd.0 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.404039+0000 front 2019-12-09T15:03:15.403202+0000 (oldest deadline 2019-12-09T15:03:58.685477+0000) 2019-12-09T15:04:50.579 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:50.573+0000 1694f700 -1 osd.1 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:14.015635+0000 front 2019-12-09T15:03:13.316714+0000 (oldest deadline 2019-12-09T15:03:59.028609+0000) 2019-12-09T15:04:50.580 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:50.575+0000 1694f700 -1 osd.1 246 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106677 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e246) 2019-12-09T15:04:50.643 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:04:50.637+0000 1694f700 -1 osd.4 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:03:15.929176+0000 front 2019-12-09T15:03:15.955288+0000 (oldest deadline 2019-12-09T15:03:58.820892+0000) 2019-12-09T15:04:50.664 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:50.659+0000 1694f700 -1 osd.5 246 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:03:32.986092+0000 (oldest deadline 2019-12-09T15:04:12.986092+0000) 2019-12-09T15:04:50.668 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:04:50.664+0000 1694f700 -1 osd.5 246 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e244) 2019-12-09T15:04:50.681 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:04:50.674+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 46sec e246 v246) 2019-12-09T15:04:51.638 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:51.634+0000 1694f700 -1 osd.1 247 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106677 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e246) 2019-12-09T15:04:52.795 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:04:52.780+0000 1694f700 -1 osd.1 248 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106677 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e246) 2019-12-09T15:05:15.717 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:05:15.709+0000 3478c700 -1 osd.2 251 build_incremental_map_msg missing incremental map 251 2019-12-09T15:05:18.577 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:05:18.566+0000 31f87700 -1 osd.2 252 build_incremental_map_msg missing incremental map 251 2019-12-09T15:06:14.720 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:14.715+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:14.735 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:14.731+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:15.025 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:15.020+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:15.702 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:15.697+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:15.745 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:15.741+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:16.016 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:16.011+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:16.146 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:37:37.606 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.162 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:37:37.622 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.178 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:37:37.638 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.184 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:37:37.644 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.192 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:37:37.674 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.198 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:37:37.680 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.425 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:37:37.885 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.446 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:37:37.928 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.465 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:37:37.925 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:16.646 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:16.640+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:16.713 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:16.708+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:16.741 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:16.737+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:16.809 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:16.803+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:16.988 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:16.983+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:17.660 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:17.656+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:17.695 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:17.690+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:17.737 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:17.733+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:17.762 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:17.757+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:17.876 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:17.871+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:17.956 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:17.951+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:18.368 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:18.362+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:18.627 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:18.622+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:18.732 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:18.728+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:18.778 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:18.774+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:18.782 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:18.777+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:18.862 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:18.858+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:18.942 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:18.937+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:19.384 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:19.378+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:19.589 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:19.584+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:19.747 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:19.739+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:19.766 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:19.762+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:19.785 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:19.782+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:19.844 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:19.840+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:19.907 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:19.901+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:20.432 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:20.421+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:20.618 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:20.612+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:20.721 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:20.716+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:20.779 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:20.774+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:20.781 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:20.776+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:20.883 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:20.871+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:20.954 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:20.948+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:21.465 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:21.459+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:21.610 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:21.605+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:21.681 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:21.676+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:21.814 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:21.808+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:21.814 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:21.809+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:21.919 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:21.913+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:21.949 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:21.944+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:22.424 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:22.418+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:22.633 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:22.627+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:22.651 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:22.645+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:22.791 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:22.786+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:22.802 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:22.798+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:22.887 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:22.883+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:22.942 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:22.937+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:23.412 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:23.405+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:23.627 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:23.622+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:23.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:23.661+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:23.792 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:23.785+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:23.806 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:23.802+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:23.897 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:23.892+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:23.930 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:23.924+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:24.375 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:24.369+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:24.620 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:24.615+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:24.649 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:24.642+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:24.780 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:24.775+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:24.798 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:24.793+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:24.885 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:24.879+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:24.894 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:24.889+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:25.333 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:25.328+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:25.615 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:06:25.608+0000 1694f700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.246530+0000 front 2019-12-09T15:05:33.445244+0000 (oldest deadline 2019-12-09T15:06:15.679375+0000) 2019-12-09T15:06:25.666 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:25.660+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:25.770 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:25.766+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:25.780 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:25.776+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:25.874 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:25.870+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:25.934 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:25.921+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:26.296 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:26.290+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:26.675 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:26.668+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:26.786 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:26.782+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:26.835 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:26.829+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:26.839 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:26.834+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:26.966 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:26.961+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:27.258 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:27.253+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:27.675 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:27.669+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:27.747 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:27.742+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:27.796 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:27.791+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:27.819 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:27.813+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:27.977 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:27.968+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:28.256 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:28.249+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:28.668 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:28.662+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:28.707 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:28.702+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:28.795 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:28.791+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:28.815 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:28.811+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:28.962 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:28.956+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:29.306 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:29.298+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:29.678 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:29.672+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:29.751 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:29.747+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:29.792 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:29.787+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:29.848 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:29.841+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:29.964 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:29.959+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:30.321 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:30.313+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:30.638 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:30.632+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:30.755 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:30.743+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:30.812 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:30.805+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:30.826 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:30.821+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:30.926 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:30.920+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:31.332 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:31.327+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:31.631 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:31.625+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:31.748 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:31.742+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:31.818 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:31.814+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:31.835 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:31.831+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:31.895 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:31.889+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:32.353 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:32.348+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:32.666 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:32.658+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:32.760 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:32.756+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:32.811 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:32.807+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:32.841 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:32.836+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:32.931 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:32.925+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:33.374 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:33.368+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:33.665 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:33.658+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:33.746 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:33.742+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:33.853 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:33.848+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:33.859 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:33.853+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:33.860 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:33.855+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:33.891 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:33.885+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:34.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:34.355+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:34.630 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:34.624+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:34.759 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:34.754+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:34.830 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:34.825+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:34.869 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:34.865+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:34.871 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:34.867+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:34.891 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:34.883+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:35.335 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:35.330+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:35.645 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:35.639+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:35.807 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:35.803+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:35.843 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:35.839+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:35.846 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:35.843+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:35.870 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:35.864+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:35.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:35.865+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:36.389 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:36.376+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:36.629 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:36.625+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:36.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:36.841+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:36.861 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:36.856+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:36.880 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:36.875+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:36.880 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:36.877+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:36.914 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:36.910+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:37.366 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:37.361+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:37.653 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:37.647+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:37.857 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:37.853+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:37.867 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:37.862+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:37.879 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:37.875+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:37.933 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:37.928+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:37.933 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:37.929+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:38.352 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:38.345+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:38.610 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:38.604+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:38.886 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:38.881+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:38.908 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:38.904+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:38.931 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:38.927+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:38.986 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:38.980+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:38.987 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:38.982+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:39.368 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:39.360+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:39.564 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:39.559+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:39.913 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:39.906+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:39.923 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:39.919+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:39.938 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:39.934+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:39.972 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:39.966+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:39.972 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:39.968+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:40.351 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:40.346+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:40.529 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:40.523+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:40.889 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:40.883+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:40.912 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:40.906+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:40.984 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:40.978+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:40.984 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:40.980+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:40.986 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:40.982+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:41.360 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:41.354+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:41.529 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:41.518+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:41.938 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:41.932+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:41.957 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:41.953+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:41.961 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:41.957+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:41.962 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:41.959+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:42.017 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:42.014+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:42.332 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:42.326+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:42.551 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:42.546+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:42.911 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:42.905+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:42.932 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:42.928+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:43.009 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:43.005+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:43.014 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:43.010+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:43.015 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:43.010+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:43.322 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:43.316+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:43.570 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:43.563+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:43.903 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:43.896+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:43.921 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:43.915+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:44.049 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:44.045+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:44.052 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:44.049+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:44.057 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:44.053+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:44.369 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:44.364+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:44.555 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:44.549+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:44.912 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:44.907+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:44.929 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:44.923+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:45.053 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:45.048+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:45.092 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:45.088+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:45.093 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:45.089+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:45.330 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:45.325+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:45.581 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:45.575+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:45.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:45.867+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:45.952 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:45.947+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:46.085 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:46.081+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:46.086 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:46.082+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:46.097 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:46.092+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:46.108 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:06:46.104+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 41sec e252 v252) 2019-12-09T15:06:46.371 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:46.364+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:46.566 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:46.561+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:46.924 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:46.918+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:46.952 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:46.948+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:47.072 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:47.067+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:47.072 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:47.068+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:47.097 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:47.093+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:47.328 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:47.322+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:47.569 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:47.559+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:47.924 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:47.919+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:47.993 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:47.988+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:48.053 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:48.050+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:48.099 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:48.095+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:48.099 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:48.096+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:48.325 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:48.319+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:48.601 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:48.594+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:48.916 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:48.910+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:49.005 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:49.001+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:49.013 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:49.008+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:49.146 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:49.139+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:49.146 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:49.141+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:49.351 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:49.345+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:49.594 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:49.588+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:49.955 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:49.949+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:50.018 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:50.014+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:50.046 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:50.040+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:50.105 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:50.099+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:50.105 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:50.101+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:50.340 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:50.335+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:50.617 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:50.612+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:50.986 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:50.981+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:51.024 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:51.020+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:51.073 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:51.069+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:51.133 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:51.130+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:51.134 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:06:51.116+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 41sec e252 v252) 2019-12-09T15:06:51.136 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:51.133+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:51.330 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:51.324+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:51.643 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:51.638+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:52.023 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:52.017+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:52.026 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:52.020+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:52.089 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:52.074+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:52.138 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:52.132+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:52.138 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:52.134+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:52.332 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:52.326+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:52.601 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:52.595+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:53.002 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:52.996+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:53.072 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:53.068+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:53.125 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:53.121+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:53.176 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:53.170+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:53.176 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:53.172+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:53.359 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:53.354+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:53.574 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:53.567+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:53.970 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:53.962+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:54.100 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:54.095+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:54.136 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:54.132+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:54.164 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:54.160+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:54.165 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:54.161+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:54.406 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:54.400+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:54.599 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:54.593+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:54.991 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:54.984+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:55.106 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:55.102+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:55.148 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:55.143+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:55.149 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:55.145+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:55.152 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:55.147+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:55.437 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:06:55.431+0000 1694f700 -1 osd.3 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:33.436621+0000 front 2019-12-09T15:05:34.102363+0000 (oldest deadline 2019-12-09T15:06:17.479540+0000) 2019-12-09T15:06:55.606 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:06:55.599+0000 1694f700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:30.154624+0000 front 2019-12-09T15:05:30.877218+0000 (oldest deadline 2019-12-09T15:06:15.956880+0000) 2019-12-09T15:06:55.981 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:06:55.975+0000 1694f700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:34.081899+0000 front 2019-12-09T15:05:33.947254+0000 (oldest deadline 2019-12-09T15:06:14.254295+0000) 2019-12-09T15:06:56.072 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:06:56.067+0000 1694f700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.878104+0000 front 2019-12-09T15:05:33.436645+0000 (oldest deadline 2019-12-09T15:06:14.068647+0000) 2019-12-09T15:06:56.166 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:06:56.161+0000 1694f700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:29.603723+0000 front 2019-12-09T15:05:29.774086+0000 (oldest deadline 2019-12-09T15:06:14.250189+0000) 2019-12-09T15:06:56.171 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:56.167+0000 1694f700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:05:32.697884+0000 front 2019-12-09T15:05:33.055045+0000 (oldest deadline 2019-12-09T15:06:17.210931+0000) 2019-12-09T15:06:56.174 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:56.171+0000 1694f700 -1 osd.4 252 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:56.186 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:06:56.171+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 41sec e252 v252) 2019-12-09T15:06:57.221 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:57.218+0000 1694f700 -1 osd.4 253 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:06:58.266 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:06:58.263+0000 1694f700 -1 osd.4 254 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:106898 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e252) 2019-12-09T15:07:40.151 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:07:40.123+0000 31f87700 -1 osd.5 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:07:40.207 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:07:40.169+0000 3578e700 -1 osd.5 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:07:40.442 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:07:40.388+0000 31f87700 -1 osd.5 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:07:40.564 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:07:40.483+0000 31786700 -1 osd.5 257 build_incremental_map_msg missing incremental map 257 2019-12-09T15:07:54.210 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:07:54.202+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:07:54.228 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:07:54.222+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:07:54.352 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:07:54.346+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:07:54.370 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:07:54.362+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:07:54.904 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:07:54.898+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:07:55.205 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:07:55.198+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:07:55.275 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:07:55.270+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:07:55.395 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:07:55.390+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:07:55.410 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:07:55.404+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:07:55.874 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:07:55.869+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:07:56.259 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:07:56.254+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:07:56.326 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:07:56.319+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:07:56.352 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:07:56.347+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:07:56.453 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:07:56.447+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:07:56.899 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:07:56.895+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:07:57.244 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:07:57.240+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:07:57.297 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:07:57.291+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:07:57.324 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:07:57.320+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:07:57.422 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:07:57.416+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:07:57.868 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:07:57.862+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:07:58.026 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:07:58.020+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:07:58.296 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:07:58.291+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:07:58.301 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:07:58.296+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:07:58.346 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:07:58.341+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:07:58.433 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:07:58.426+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:07:58.869 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:07:58.864+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:07:59.013 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:07:59.007+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:07:59.292 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:07:59.286+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:07:59.334 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:07:59.329+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:07:59.395 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:07:59.390+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:07:59.449 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:07:59.444+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:07:59.905 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:07:59.900+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:07:59.983 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:07:59.975+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:00.279 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:00.272+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:00.355 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:00.351+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:00.359 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:00.355+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:00.491 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:00.480+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:00.891 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:00.886+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:00.987 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:00.979+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:00.988 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:00.982+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:01.279 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:01.271+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:01.371 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:01.366+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:01.392 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:01.388+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:01.445 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:01.439+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:01.909 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:01.904+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:01.940 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:01.933+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:01.942 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:01.937+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:02.257 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:02.250+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:02.358 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:02.353+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:02.393 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:02.388+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:02.457 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:02.451+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:02.898 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:02.891+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:02.982 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:02.976+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:02.984 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:02.979+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:03.247 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:03.240+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:03.342 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:03.336+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:03.380 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:03.374+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:03.487 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:03.481+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:03.935 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:03.929+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:04.026 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:04.018+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:04.030 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:04.024+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:04.285 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:04.279+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:04.333 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:04.327+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:04.343 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:04.338+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:04.493 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:04.485+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:04.968 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:04.962+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:05.013 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:05.006+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:05.014 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:05.009+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:05.256 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:05.245+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:05.312 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:05.301+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:05.338 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:05.333+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:05.498 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:05.492+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:05.928 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:05.923+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:06.019 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:06.010+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:06.020 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:06.013+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:06.265 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:06.259+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:06.272 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:06.267+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:06.330 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:06.324+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:06.540 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:06.532+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:06.962 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:06.957+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:07.059 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:07.052+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:07.060 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:07.054+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:07.276 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:07.270+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:07.308 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:07.301+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:07.322 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:07.316+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:07.550 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:07.542+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:07.967 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:07.961+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:08.037 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:08.029+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:08.038 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:08.031+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:08.245 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:08.239+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:08.351 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:08.343+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:08.360 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:08.354+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:08.547 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:08.539+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:08.980 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:08.974+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:08.998 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:08.991+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:08.999 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:08.993+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:09.292 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:09.287+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:09.342 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:09.334+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:09.356 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:09.349+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:09.575 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:09.569+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:09.958 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:09.951+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:10.011 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:10.003+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:10.013 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:10.007+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:10.260 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:10.256+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:10.279 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:10.273+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:10.302 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:10.296+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:10.402 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:10.392+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:10.555 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:10.548+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:10.958 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:10.952+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:11.059 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:11.052+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:11.062 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:11.056+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:11.277 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:11.271+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:11.304 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:11.294+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:11.327 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:11.322+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:11.394 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:11.388+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:11.587 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:11.580+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:12.001 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:11.995+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:12.069 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:12.062+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:12.071 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:12.063+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:12.310 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:12.306+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:12.312 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:12.307+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:12.313 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:12.308+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:12.349 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:12.343+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:12.544 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:12.537+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:13.011 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:13.006+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:13.045 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:13.037+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:13.046 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:13.040+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:13.277 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:13.272+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:13.320 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:13.316+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:13.348 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:13.342+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:13.359 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:13.354+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:13.501 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:13.495+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:13.994 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:13.989+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:14.045 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:14.039+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:14.047 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:14.042+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:14.318 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:14.313+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:14.327 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:14.321+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:14.356 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:14.349+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:14.400 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:14.395+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:14.514 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:14.507+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:15.028 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:15.022+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:15.029 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:15.023+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:15.049 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:15.043+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:15.313 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:15.306+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:15.348 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:15.343+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:15.380 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:15.373+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:15.428 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:15.422+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:15.470 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:15.463+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:16.046 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:16.038+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:16.047 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:16.040+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:16.101 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:16.097+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:16.317 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:16.312+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:16.335 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:16.329+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:16.386 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:16.378+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:16.441 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:16.435+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:16.442 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:16.436+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:17.078 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:17.073+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:17.079 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:17.073+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:17.080 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:17.076+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:17.301 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:17.295+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:17.303 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:17.298+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:17.339 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:17.333+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:17.418 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:17.413+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:17.447 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:17.442+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:17.480 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:17.475+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:18.045 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:18.035+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:18.045 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:18.038+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:18.106 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:18.102+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:18.268 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:18.261+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:18.269 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:18.264+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:18.322 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:18.316+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:18.407 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:18.402+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:18.419 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:18.414+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:18.514 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:18.509+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:19.000 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:18.994+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:19.001 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:18.996+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:19.104 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:19.099+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:19.266 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:19.261+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:19.270 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:19.266+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:19.354 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:19.349+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:19.429 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:19.422+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:19.449 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:19.442+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:19.528 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:19.520+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:19.984 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:19.978+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:19.986 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:19.981+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:20.124 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:20.118+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:20.287 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:20.279+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:20.288 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:20.283+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:20.354 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:20.348+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:20.388 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:20.383+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:20.494 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:20.489+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:20.550 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:20.544+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:20.947 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:20.940+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:20.948 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:20.943+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:21.111 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:21.106+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:21.271 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:21.265+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:21.272 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:21.267+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:21.340 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:21.334+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:21.398 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:21.392+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:21.499 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:21.494+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:21.506 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:21.501+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:21.927 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:21.921+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:21.927 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:21.922+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:22.104 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:22.099+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:22.277 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:22.272+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:22.278 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:22.274+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:22.369 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:22.364+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:22.370 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:22.363+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:22.478 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:22.472+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:22.518 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:22.512+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:22.946 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:22.940+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:22.946 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:22.942+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:23.059 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:23.054+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:23.322 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:23.315+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:23.322 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:23.318+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:23.336 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:23.331+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:23.416 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:23.407+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:23.526 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:23.519+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:23.561 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:23.553+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:23.925 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:23.918+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:23.928 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:23.923+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:24.094 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:24.087+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:24.367 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:24.362+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:24.372 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:24.367+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:24.372 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:24.366+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:24.398 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:24.392+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:24.483 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:24.477+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:24.613 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:24.608+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:24.893 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:24.886+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:24.904 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:24.900+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:25.087 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:25.081+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:25.394 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:25.389+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:25.407 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:25.402+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:25.411 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:25.407+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:25.448 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:25.442+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:25.529 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:25.520+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:25.617 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:25.612+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:25.934 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:25.926+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:25.935 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:25.929+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:26.069 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:26.063+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:26.421 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:26.414+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:26.421 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:26.416+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:26.438 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:26.432+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:26.507 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:26.495+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:26.532 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:26.526+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:26.597 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:08:26.591+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 41sec e258 v258) 2019-12-09T15:08:26.673 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:26.665+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:26.983 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:26.977+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:26.984 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:26.979+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:27.085 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:27.080+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:27.397 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:27.391+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:27.398 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:27.394+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:27.473 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:27.467+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:27.478 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:27.472+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:27.546 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:27.541+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:27.686 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:27.681+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:28.015 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:28.010+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:28.015 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:28.011+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:28.122 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:28.117+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:28.361 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:28.354+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:28.362 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:28.357+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:28.465 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:28.457+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:28.482 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:28.476+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:28.587 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:28.582+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:28.719 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:28.714+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:28.986 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:28.981+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:28.987 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:28.983+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:29.157 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:29.152+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:29.360 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:29.354+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:29.361 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:29.357+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:29.479 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:29.474+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:29.484 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:29.478+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:29.550 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:29.545+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:29.744 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:29.737+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:30.025 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:30.018+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:30.026 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:30.021+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:30.151 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:30.146+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:30.340 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:30.332+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:30.344 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:30.340+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:30.453 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:30.446+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:30.497 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:30.491+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:30.532 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:30.526+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:30.781 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:30.776+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:31.046 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:31.041+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:31.047 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:31.043+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:31.168 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:31.162+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:31.366 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:31.362+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:31.370 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:31.366+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:31.467 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:31.461+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:31.509 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:31.503+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:31.518 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:31.513+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:31.620 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:08:31.613+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 41sec e258 v258) 2019-12-09T15:08:31.749 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:31.744+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:32.013 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:32.006+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:32.013 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:32.008+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:32.210 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:32.205+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:32.391 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:32.384+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:32.392 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:32.387+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:32.448 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:32.443+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:32.483 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:32.478+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:32.550 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:32.544+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:32.705 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:32.700+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:33.027 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:33.022+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:33.028 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:33.023+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:33.169 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:33.163+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:33.400 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:33.394+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:33.401 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:33.397+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:33.421 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:33.415+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:33.462 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:33.456+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:33.528 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:33.521+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:33.677 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:33.672+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:34.032 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:34.027+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:34.034 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:34.030+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:34.186 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:34.181+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:34.384 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:34.378+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:34.385 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:34.380+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:34.463 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:34.458+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:34.514 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:34.508+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:34.574 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:34.569+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:34.703 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:34.698+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:35.030 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:35.023+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:35.031 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:35.026+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:35.223 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:35.218+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:35.407 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:35.401+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:35.412 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:35.403+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:35.447 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:35.442+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:35.485 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:35.478+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:35.578 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:35.571+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:35.738 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:35.731+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:36.001 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:35.993+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:36.001 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:35.995+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:36.219 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:36.215+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:36.387 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:36.380+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:36.389 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:36.383+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:36.446 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:36.440+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:36.519 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:36.511+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:36.535 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:36.529+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:36.628 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:08:36.623+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 41sec e258 v258) 2019-12-09T15:08:36.751 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:36.745+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:36.955 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:36.949+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:36.956 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:36.951+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:37.239 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:37.234+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:37.380 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:37.374+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:37.380 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:37.376+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:37.491 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:37.485+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:37.501 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:37.496+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:37.521 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:37.515+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:37.770 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:37.765+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:37.933 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:37.927+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:37.934 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:37.929+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:38.290 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:38.286+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:38.420 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:38.414+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:38.421 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:38.417+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:38.459 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:38.453+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:38.508 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:38.502+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:38.543 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:38.536+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:38.812 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:38.806+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:38.921 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:38.913+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:38.922 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:38.916+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:39.292 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:39.286+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:39.420 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:39.415+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:39.428 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:39.420+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:39.466 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:39.461+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:39.472 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:39.467+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:39.573 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:39.567+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:39.845 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:39.840+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:39.934 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:39.928+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:39.934 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:39.930+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:40.297 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:40.292+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:40.416 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:40.410+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:40.417 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:40.412+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:40.436 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:40.430+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:40.481 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:40.469+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:40.571 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:40.563+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:40.869 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:40.861+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:40.920 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:40.913+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:40.923 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:40.918+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:41.290 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:41.284+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:41.388 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:41.382+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:41.389 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:41.385+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:41.430 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:41.424+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:41.524 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:41.518+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:41.541 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:41.527+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:41.641 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:08:41.636+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 41sec e258 v258) 2019-12-09T15:08:41.895 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:41.890+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:41.897 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:41.893+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:41.900 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:41.894+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:42.260 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:42.255+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:42.370 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:42.357+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:42.373 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:42.360+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:42.401 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:42.396+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:42.480 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:42.475+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:42.517 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:42.512+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:42.922 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:42.917+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:42.929 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:42.921+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:42.930 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:42.924+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:43.220 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:43.214+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:43.398 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:43.392+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:43.399 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:43.395+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:43.414 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:43.409+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:43.474 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:43.468+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:43.504 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:43.499+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:43.933 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:43.928+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:43.936 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:43.932+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:43.978 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:43.971+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:44.237 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:44.231+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:44.384 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:44.378+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:44.385 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:44.381+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:44.428 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:44.422+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:44.449 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:44.444+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:44.479 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:44.474+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:44.934 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:44.928+0000 1694f700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.345578+0000 front 2019-12-09T15:07:12.555309+0000 (oldest deadline 2019-12-09T15:07:57.617315+0000) 2019-12-09T15:08:44.934 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:44.929+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:44.998 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:44.994+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:45.215 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:45.209+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:45.391 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:45.387+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:45.394 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:45.390+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:45.446 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:45.441+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:45.453 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:45.448+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:45.508 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:45.503+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:45.929 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:45.923+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:45.961 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:45.951+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:46.212 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:46.207+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:46.368 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:46.363+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:46.372 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:46.368+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:46.451 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:46.445+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:46.455 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:46.448+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:46.499 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:46.494+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:46.658 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:08:46.653+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 41sec e258 v258) 2019-12-09T15:08:46.929 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:46.924+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:46.990 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:46.985+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:47.206 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:47.198+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:47.364 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:47.358+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:47.365 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:47.360+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:47.464 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:47.458+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:47.499 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:47.494+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:47.520 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:47.514+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:47.911 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:47.905+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:47.970 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:47.964+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:48.228 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:48.223+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:48.381 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:48.373+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:48.382 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:48.376+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:48.469 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:48.463+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:48.492 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:48.487+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:48.561 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:48.555+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:48.871 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:48.867+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:48.952 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:48.946+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:49.270 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:49.265+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:49.399 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:49.387+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:49.404 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:49.389+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:49.489 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:49.483+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:49.492 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:49.487+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:49.561 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:49.555+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:49.905 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:49.900+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:49.939 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:49.934+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:50.291 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:50.286+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:50.371 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:50.364+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:50.374 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:50.371+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:50.458 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:50.452+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:50.466 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:50.459+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:50.532 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:50.527+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:50.890 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:50.885+0000 1694f700 -1 osd.1 258 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:50.942 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:08:50.937+0000 1694f700 -1 osd.6 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:07.579446+0000 front 2019-12-09T15:07:07.657962+0000 (oldest deadline 2019-12-09T15:07:53.318116+0000) 2019-12-09T15:08:51.290 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:08:51.286+0000 1694f700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.499772+0000 front 2019-12-09T15:07:08.511145+0000 (oldest deadline 2019-12-09T15:07:54.338729+0000) 2019-12-09T15:08:51.396 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:51.391+0000 1694f700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:07:36.467555+0000 (oldest deadline 2019-12-09T15:08:16.467555+0000) 2019-12-09T15:08:51.397 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:51.394+0000 1694f700 -1 osd.5 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-12-09T15:08:51.414 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:08:51.408+0000 1694f700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:09.351214+0000 front 2019-12-09T15:07:09.427851+0000 (oldest deadline 2019-12-09T15:07:53.329926+0000) 2019-12-09T15:08:51.465 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:08:51.460+0000 1694f700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:08.182014+0000 front 2019-12-09T15:07:08.189316+0000 (oldest deadline 2019-12-09T15:07:54.055077+0000) 2019-12-09T15:08:51.507 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:08:51.502+0000 1694f700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:07:12.266699+0000 front 2019-12-09T15:07:12.256164+0000 (oldest deadline 2019-12-09T15:07:53.947330+0000) 2019-12-09T15:08:51.700 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:08:51.694+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 41sec e258 v258) 2019-12-09T15:08:52.126 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:52.008+0000 1694f700 -1 osd.1 259 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e254) 2019-12-09T15:08:52.219 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:08:52.095+0000 32788700 -1 osd.5 259 build_incremental_map_msg missing incremental map 257 2019-12-09T15:08:52.987 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:08:52.980+0000 1694f700 -1 osd.1 259 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107082 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat,read 0~4194304] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e258) 2019-12-09T15:08:58.906 INFO:teuthology.orchestra.run.smithi033.stderr:testIfMatch (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:09:10.222 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:09:10.165+0000 3478c700 -1 osd.4 262 build_incremental_map_msg missing incremental map 262 2019-12-09T15:09:19.344 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:09:19.290+0000 30f85700 -1 osd.5 263 build_incremental_map_msg missing incremental map 257 2019-12-09T15:09:36.053 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:09:36.046+0000 31786700 -1 osd.4 263 build_incremental_map_msg missing incremental map 262 2019-12-09T15:09:57.002 INFO:teuthology.orchestra.run.smithi033.stderr:testIfMatchAndUnmodified (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:10:09.609 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:41:31.068 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.614 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:41:31.074 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.633 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:41:31.087 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.634 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:41:31.090 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.634 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:41:31.107 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.634 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:41:31.111 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.859 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:41:31.340 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.961 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:41:31.421 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:09.984 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:41:31.443 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:12.482 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:12.477+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:12.582 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:12.575+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:12.859 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:12.855+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:13.513 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:13.507+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:13.616 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:13.610+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:13.858 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:13.852+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:13.858 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:13.855+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:14.493 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:14.487+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:14.609 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:14.593+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:14.838 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:14.833+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:14.862 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:14.851+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:15.500 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:15.495+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:15.580 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:15.576+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:15.593 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:15.587+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:15.814 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:15.807+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:15.825 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:15.820+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:15.848 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:15.841+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:16.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:16.489+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:16.613 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:16.604+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:16.639 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:16.632+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:16.846 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:16.839+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:16.852 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:16.848+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:16.862 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:16.849+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:17.529 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:17.525+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:17.590 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:17.586+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:17.665 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:17.657+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:17.810 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:17.806+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:17.878 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:17.871+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:17.880 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:17.873+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:18.536 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:18.533+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:18.620 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:18.616+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:18.640 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:18.633+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:18.857 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:18.852+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:18.883 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:18.875+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:18.893 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:18.880+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:19.501 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:19.496+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:19.579 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:19.575+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:19.622 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:19.616+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:19.813 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:19.808+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:19.871 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:19.859+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:19.898 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:19.892+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:20.509 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:20.504+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:20.562 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:20.557+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:20.652 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:20.646+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:20.852 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:20.848+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:20.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:20.861+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:20.903 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:20.897+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:21.523 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:21.517+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:21.556 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:21.553+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:21.619 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:21.615+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:21.853 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:21.848+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:21.899 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:21.893+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:21.926 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:21.920+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:22.555 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:22.550+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:22.605 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:22.600+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:22.670 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:22.663+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:22.814 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:22.810+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:22.887 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:22.880+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:22.942 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:22.936+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:23.482 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:23.476+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:23.591 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:23.587+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:23.627 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:23.622+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:23.689 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:23.684+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:23.783 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:23.779+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:23.933 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:23.926+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:23.948 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:23.942+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:24.497 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:24.489+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:24.611 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:24.606+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:24.621 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:24.617+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:24.674 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:24.668+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:24.770 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:24.767+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:24.959 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:24.950+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:24.959 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:24.941+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:25.482 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:25.477+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:25.592 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:25.587+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:25.641 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:25.637+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:25.717 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:25.711+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:25.749 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:25.742+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:25.954 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:25.949+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:26.005 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:26.000+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:26.490 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:26.484+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:26.621 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:26.616+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:26.691 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:26.685+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:26.727 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:26.723+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:26.741 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:26.735+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:26.965 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:26.959+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:26.971 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:26.965+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:27.459 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:27.452+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:27.638 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:27.633+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:27.703 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:27.699+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:27.709 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:27.704+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:27.781 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:27.776+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:27.953 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:27.947+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:27.958 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:27.950+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:28.472 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:28.467+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:28.683 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:28.680+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:28.696 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:28.692+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:28.736 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:28.731+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:28.765 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:28.760+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:28.947 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:28.941+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:28.983 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:28.976+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:29.497 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:29.488+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:29.654 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:29.648+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:29.655 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:29.650+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:29.685 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:29.681+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:29.769 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:29.765+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:29.811 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:29.805+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:29.948 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:29.943+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:29.988 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:29.981+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:30.522 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:30.516+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:30.642 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:30.637+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:30.643 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:30.638+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:30.715 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:30.710+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:30.803 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:30.797+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:30.853 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:30.846+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:30.997 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:30.991+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:31.006 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:30.996+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:31.501 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:31.495+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:31.616 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:31.611+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:31.616 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:31.612+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:31.688 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:31.684+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:31.783 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:31.776+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:31.829 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:31.822+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:32.026 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:32.020+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:32.033 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:32.026+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:32.459 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:32.451+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:32.653 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:32.648+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:32.654 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:32.650+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:32.700 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:32.692+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:32.752 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:32.748+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:32.827 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:32.820+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:32.991 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:32.985+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:33.015 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:33.010+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:33.474 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:33.468+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:33.653 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:33.648+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:33.654 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:33.649+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:33.749 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:33.745+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:33.749 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:33.744+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:33.860 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:33.854+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:33.997 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:33.991+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:34.013 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:34.006+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:34.455 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:34.449+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:34.634 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:34.629+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:34.635 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:34.631+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:34.742 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:34.736+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:34.793 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:34.788+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:34.862 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:34.856+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:34.987 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:34.981+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:35.017 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:35.012+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:35.461 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:35.455+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:35.685 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:35.680+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:35.686 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:35.681+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:35.779 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:35.775+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:35.828 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:35.820+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:35.844 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:35.833+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:35.965 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:35.960+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:36.004 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:35.999+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:36.500 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:36.494+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:36.712 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:36.708+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:36.712 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:36.709+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:36.814 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:36.811+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:36.848 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:36.846+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:36.874 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:36.868+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:37.001 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:36.995+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:37.029 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:37.023+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:37.516 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:37.510+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:37.729 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:37.724+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:37.729 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:37.726+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:37.771 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:37.768+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:37.867 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:37.855+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:37.921 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:37.915+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:37.959 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:37.953+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:38.050 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:38.045+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:38.559 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:38.553+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:38.719 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:38.715+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:38.722 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:38.719+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:38.768 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:38.765+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:38.871 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:38.868+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:38.967 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:38.961+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:38.972 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:38.966+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:39.091 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:39.084+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:39.579 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:39.574+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:39.698 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:39.694+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:39.698 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:39.696+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:39.756 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:39.753+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:39.870 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:39.863+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:39.976 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:39.970+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:39.984 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:39.972+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:40.059 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:40.053+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:40.631 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:40.625+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:40.671 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:40.660+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:40.672 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:40.661+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:40.733 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:40.730+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:40.913 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:40.909+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:40.945 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:40.939+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:41.001 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:40.995+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:41.101 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:41.095+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:41.661 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:41.656+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:41.713 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:41.710+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:41.713 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:41.711+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:41.754 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:41.750+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:41.884 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:41.880+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:41.956 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:41.950+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:42.028 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:42.023+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:42.095 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:42.090+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:42.715 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:42.708+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:42.716 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:42.711+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:42.716 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:42.712+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:42.737 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:42.733+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:42.840 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:42.836+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:42.969 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:42.963+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:43.007 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:43.001+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:43.066 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:43.060+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:43.676 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:43.670+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:43.760 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:43.716+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:43.760 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:43.718+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:43.761 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:43.723+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:43.795 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:43.790+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:43.936 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:43.930+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:44.018 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:44.012+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:44.039 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:44.034+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:44.647 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:44.642+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:44.733 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:44.729+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:44.769 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:44.765+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:44.769 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:44.766+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:44.846 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:44.841+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:44.924 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:44.918+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:45.025 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:45.020+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:45.029 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:45.024+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:45.619 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:45.612+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:45.704 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:45.699+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:45.765 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:45.760+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:45.766 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:45.761+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:45.839 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:45.834+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:45.954 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:45.948+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:46.045 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:46.040+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:46.051 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:46.045+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:46.611 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:46.605+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:46.742 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:46.737+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:46.743 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:46.738+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:46.748 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:46.744+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:46.884 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:46.880+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:46.952 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:46.943+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:47.015 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:47.009+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:47.021 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:47.016+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:47.091 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:10:47.081+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e263 v263) 2019-12-09T15:10:47.635 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:47.630+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:47.769 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:47.765+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:47.771 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:47.768+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:47.777 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:47.771+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:47.924 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:47.920+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:47.929 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:47.924+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:48.045 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:48.039+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:48.060 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:48.054+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:48.619 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:48.614+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:48.774 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:48.767+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:48.775 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:48.770+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:48.822 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:48.818+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:48.946 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:48.940+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:48.969 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:48.964+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:49.034 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:49.028+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:49.075 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:49.070+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:49.633 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:49.627+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:49.755 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:49.751+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:49.757 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:49.754+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:49.843 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:49.837+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:49.977 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:49.970+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:49.978 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:49.972+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:50.033 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:50.027+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:50.034 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:50.027+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:50.685 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:50.679+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:50.799 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:50.794+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:50.800 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:50.796+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:50.801 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:50.798+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:51.020 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:51.012+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:51.027 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:51.023+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:51.035 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:51.029+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:51.057 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:51.051+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:51.733 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:10:51.728+0000 1694f700 -1 osd.1 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:26.604948+0000 front 2019-12-09T15:09:26.619853+0000 (oldest deadline 2019-12-09T15:10:23.371517+0000) 2019-12-09T15:10:51.780 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:10:51.776+0000 1694f700 -1 osd.7 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.462674+0000 front 2019-12-09T15:09:28.934257+0000 (oldest deadline 2019-12-09T15:10:12.204080+0000) 2019-12-09T15:10:51.781 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:51.776+0000 1694f700 -1 osd.4 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.970888+0000 front 2019-12-09T15:09:28.531779+0000 (oldest deadline 2019-12-09T15:10:11.962189+0000) 2019-12-09T15:10:51.782 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:51.777+0000 1694f700 -1 osd.4 263 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:51.994 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:10:51.988+0000 1694f700 -1 osd.0 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:29.729013+0000 front 2019-12-09T15:09:29.727387+0000 (oldest deadline 2019-12-09T15:10:15.086950+0000) 2019-12-09T15:10:52.031 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:10:52.026+0000 1694f700 -1 osd.6 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:27.965996+0000 front 2019-12-09T15:09:27.970937+0000 (oldest deadline 2019-12-09T15:10:12.551905+0000) 2019-12-09T15:10:52.050 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:10:52.044+0000 1694f700 -1 osd.2 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:28.531495+0000 front 2019-12-09T15:09:27.969845+0000 (oldest deadline 2019-12-09T15:10:13.779535+0000) 2019-12-09T15:10:52.051 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:10:52.045+0000 1694f700 -1 osd.3 263 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:09:30.046854+0000 front 2019-12-09T15:09:31.592730+0000 (oldest deadline 2019-12-09T15:10:14.973068+0000) 2019-12-09T15:10:52.142 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:10:52.133+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e263 v263) 2019-12-09T15:10:52.865 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:52.858+0000 1694f700 -1 osd.4 264 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:10:53.957 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:10:53.954+0000 1694f700 -1 osd.4 265 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107478 2.23s0 2:c417719b:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_c8110d165fb544bebcb2d9ffdcd436bb:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e263) 2019-12-09T15:11:07.552 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:42:29.011 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.594 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:42:29.054 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.636 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:42:29.095 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.638 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:42:29.097 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.652 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:42:29.134 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.660 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:42:29.142 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.678 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:42:29.138 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.752 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:42:29.212 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:07.802 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:42:29.284 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:11:49.522 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:49.515+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:49.916 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:49.911+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:50.502 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:50.497+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:50.543 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:50.537+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:50.898 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:50.891+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:51.466 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:51.460+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:51.579 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:51.574+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:51.928 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:51.924+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:52.461 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:52.455+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:52.606 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:52.600+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:52.887 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:52.881+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:53.467 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:53.461+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:53.580 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:11:53.574+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:11:53.618 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:53.607+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:53.905 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:53.899+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:54.302 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:11:54.296+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:11:54.461 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:54.455+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:54.572 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:11:54.565+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:11:54.604 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:54.599+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:54.888 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:54.882+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:55.297 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:11:55.290+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:11:55.459 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:55.452+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:55.539 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:11:55.533+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:11:55.610 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:55.604+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:55.611 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:55.606+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:11:55.901 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:55.896+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:56.289 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:11:56.284+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:11:56.498 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:56.493+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:56.528 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:11:56.522+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:11:56.609 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:56.602+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:56.610 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:56.604+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:11:56.858 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:56.852+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:57.281 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:11:57.275+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:11:57.502 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:57.496+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:57.562 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:11:57.557+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:11:57.587 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:57.581+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:57.588 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:57.583+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:11:57.888 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:57.881+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:58.324 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:11:58.307+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:11:58.465 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:58.459+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:58.589 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:11:58.573+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:11:58.611 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:58.604+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:58.612 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:58.607+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:11:58.919 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:58.914+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:11:59.341 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:11:59.335+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:11:59.503 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:11:59.498+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:11:59.571 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:11:59.565+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:11:59.623 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:59.618+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:11:59.624 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:11:59.619+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:11:59.949 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:11:59.943+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:00.326 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:00.320+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:00.518 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:00.510+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:00.534 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:00.527+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:00.661 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:00.654+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:00.662 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:00.657+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:00.941 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:00.935+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:01.142 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:01.137+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:01.357 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:01.351+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:01.503 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:01.497+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:01.514 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:01.507+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:01.690 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:01.685+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:01.690 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:01.686+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:01.911 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:01.905+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:02.117 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:02.113+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:02.411 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:02.405+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:02.495 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:02.490+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:02.504 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:02.499+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:02.676 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:02.670+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:02.676 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:02.671+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:02.899 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:02.893+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:03.164 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:03.159+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:03.428 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:03.422+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:03.518 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:03.513+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:03.525 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:03.519+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:03.713 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:03.708+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:03.715 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:03.711+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:03.923 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:03.917+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:04.155 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:04.150+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:04.470 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:04.464+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:04.541 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:04.536+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:04.547 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:04.542+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:04.766 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:04.757+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:04.767 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:04.760+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:04.948 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:04.944+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:05.168 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:05.163+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:05.506 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:05.501+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:05.519 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:05.507+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:05.582 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:05.577+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:05.739 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:05.727+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:05.740 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:05.729+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:05.950 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:05.946+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:06.148 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:06.143+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:06.511 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:06.502+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:06.551 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:06.544+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:06.561 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:06.556+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:06.766 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:06.759+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:06.769 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:06.763+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:07.002 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:06.996+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:07.195 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:07.189+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:07.202 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:07.196+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:07.528 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:07.520+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:07.557 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:07.550+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:07.565 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:07.557+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:07.738 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:07.731+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:07.739 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:07.733+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:08.018 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:08.012+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:08.223 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:08.217+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:08.227 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:08.222+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:08.557 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:08.550+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:08.559 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:08.553+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:08.561 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:08.556+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:08.693 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:08.686+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:08.693 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:08.687+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:09.047 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:09.041+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:09.217 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:09.211+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:09.218 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:09.213+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:09.542 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:09.535+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:09.580 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:09.575+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:09.602 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:09.594+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:09.651 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:09.644+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:09.651 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:09.646+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:10.088 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:10.083+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:10.258 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:10.251+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:10.259 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:10.254+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:10.500 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:10.493+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:10.591 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:10.583+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:10.631 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:10.626+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:10.672 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:10.665+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:10.672 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:10.666+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:11.068 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:11.061+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:11.259 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:11.253+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:11.261 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:11.256+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:11.517 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:11.510+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:11.627 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:11.621+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:11.639 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:11.630+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:11.719 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:11.710+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:11.719 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:11.714+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:12.115 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:12.109+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:12.299 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:12.294+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:12.303 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:12.299+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:12.512 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:12.505+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:12.636 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:12.631+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:12.655 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:12.647+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:12.773 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:12.765+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:12.773 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:12.767+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:13.116 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:13.110+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:13.309 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:13.302+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:13.310 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:13.305+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:13.486 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:13.480+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:13.594 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:13.589+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:13.685 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:13.679+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:13.820 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:13.813+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:13.821 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:13.815+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:14.094 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:14.087+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:14.269 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:14.264+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:14.273 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:14.268+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:14.519 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:14.512+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:14.569 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:14.563+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:14.727 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:14.720+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:14.819 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:14.811+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:14.819 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:14.813+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:15.085 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:15.079+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:15.272 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:15.267+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:15.276 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:15.272+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:15.548 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:15.540+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:15.551 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:15.546+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:15.766 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:15.758+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:15.805 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:15.799+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:15.808 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:15.802+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:16.114 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:16.108+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:16.294 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:16.288+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:16.295 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:16.290+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:16.562 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:16.554+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:16.599 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:16.594+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:16.810 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:16.803+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:16.810 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:16.805+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:16.813 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:16.806+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:17.089 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:17.083+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:17.322 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:17.317+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:17.326 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:17.322+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:17.565 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:17.558+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:17.647 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:17.641+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:17.781 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:17.775+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:17.782 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:17.777+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:17.860 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:17.854+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:18.047 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:18.042+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:18.370 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:18.363+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:18.371 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:18.366+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:18.576 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:18.568+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:18.693 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:18.687+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:18.805 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:18.798+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:18.807 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:18.800+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:18.830 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:18.824+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:19.087 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:19.081+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:19.337 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:19.332+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:19.341 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:19.337+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:19.570 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:19.564+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:19.662 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:19.657+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:19.671 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:19.667+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:19.768 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:19.762+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:19.768 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:19.763+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:19.787 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:19.782+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:20.139 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:20.132+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:20.352 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:20.343+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:20.352 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:20.345+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:20.617 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:20.610+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:20.711 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:20.706+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:20.715 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:20.709+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:20.751 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:20.740+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:20.761 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:20.744+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:20.840 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:20.834+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:21.099 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:21.093+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:21.305 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:21.300+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:21.306 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:21.302+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:21.642 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:21.635+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:21.672 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:21.666+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:21.704 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:21.695+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:21.792 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:21.785+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:21.793 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:21.788+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:21.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:21.842+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:22.057 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:22.051+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:22.280 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:22.273+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:22.281 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:22.277+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:22.628 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:22.621+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:22.639 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:22.632+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:22.698 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:22.691+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:22.791 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:22.784+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:22.795 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:22.789+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:22.810 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:12:22.804+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e269 v269) 2019-12-09T15:12:22.823 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:22.817+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:23.066 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:23.061+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:23.272 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:23.266+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:23.273 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:23.268+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:23.661 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:23.656+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:23.671 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:23.665+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:23.696 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:23.691+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:23.809 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:23.804+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:23.810 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:23.803+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:23.810 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:23.806+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:24.044 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:24.037+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:24.304 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:24.298+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:24.308 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:24.305+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:24.642 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:24.636+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:24.651 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:24.646+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:24.688 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:24.684+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:24.776 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:24.767+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:24.777 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:24.770+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:24.865 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:24.856+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:25.027 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:25.023+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:25.331 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:25.327+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:25.334 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:25.331+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:25.662 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:25.656+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:25.672 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:25.667+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:25.698 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:25.691+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:25.805 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:25.798+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:25.805 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:25.800+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:25.838 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:25.832+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:26.067 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:26.062+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:26.386 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:26.382+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:26.390 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:26.387+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:26.698 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:26.694+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:26.699 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:26.692+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:26.735 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:26.731+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:26.833 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:26.827+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:26.833 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:26.828+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:26.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:26.841+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:27.095 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:27.090+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:27.360 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:27.356+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:27.364 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:27.361+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:27.717 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:27.713+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:27.718 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:27.712+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:27.753 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:27.749+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:27.812 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:27.806+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:27.813 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:27.809+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:27.898 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:27.893+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:27.948 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:12:27.943+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e269 v269) 2019-12-09T15:12:27.978 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:43:49.438 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.032 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:43:49.492 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.064 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:43:49.546 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.069 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:43:49.551 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.090 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:28.085+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:28.140 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:43:49.599 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.145 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:43:49.604 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.147 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:43:49.606 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.187 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:43:49.669 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.276 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:43:49.736 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:12:28.417 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:28.411+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:28.417 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:28.414+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:28.720 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:28.712+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:28.749 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:28.745+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:28.761 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:28.756+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:28.779 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:28.774+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:28.779 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:28.775+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:28.943 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:28.938+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:29.071 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:29.068+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:29.439 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:29.435+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:29.439 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:29.437+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:29.758 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:29.753+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:29.776 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:29.772+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:29.786 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:29.780+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:29.831 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:29.824+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:29.831 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:29.827+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:29.939 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:29.931+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:30.082 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:30.078+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:30.444 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:30.438+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:30.445 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:30.441+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:30.742 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:30.735+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:30.758 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:30.752+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:30.800 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:30.792+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:30.882 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:30.876+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:30.884 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:30.878+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:30.920 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:30.913+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:31.037 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:31.031+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:31.483 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:31.478+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:31.487 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:31.483+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:31.696 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:31.690+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:31.767 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:31.762+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:31.778 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:31.772+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:31.836 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:31.830+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:31.837 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:31.832+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:31.967 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:31.961+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:31.996 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:31.991+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:32.509 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:32.503+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:32.510 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:32.506+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:32.726 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:32.721+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:32.740 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:32.734+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:32.822 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:32.816+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:32.838 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:32.823+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:32.839 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:32.825+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:32.946 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:32.939+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:32.968 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:12:32.962+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e269 v269) 2019-12-09T15:12:33.004 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:32.999+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:33.469 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:33.462+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:33.470 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:33.465+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:33.728 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:33.723+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:33.756 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:33.750+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:33.789 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:33.784+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:33.829 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:33.823+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:33.831 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:33.827+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:33.927 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:33.921+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:34.043 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:34.036+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:34.504 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:34.497+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:34.505 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:34.500+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:34.735 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:34.730+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:34.742 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:34.736+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:34.789 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:34.783+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:34.857 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:34.851+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:34.860 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:34.855+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:34.957 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:34.951+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:35.058 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:35.053+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:35.503 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:35.498+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:35.504 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:35.500+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:35.697 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:35.692+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:35.754 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:35.748+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:35.815 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:35.809+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:35.816 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:35.811+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:35.833 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:35.826+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:36.006 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:35.999+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:36.019 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:36.012+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:36.503 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:36.498+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:36.507 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:36.503+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:36.685 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:36.679+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:36.732 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:36.727+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:36.785 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:36.778+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:36.785 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:36.779+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:36.841 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:36.836+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:36.995 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:36.988+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:37.030 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:37.024+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:37.460 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:37.455+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:37.463 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:37.460+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:37.704 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:12:37.699+0000 1694f700 -1 osd.3 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.764016+0000 front 2019-12-09T15:11:06.713423+0000 (oldest deadline 2019-12-09T15:11:49.575529+0000) 2019-12-09T15:12:37.711 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:12:37.705+0000 1694f700 -1 osd.7 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:06.475398+0000 front 2019-12-09T15:11:06.196085+0000 (oldest deadline 2019-12-09T15:12:19.353307+0000) 2019-12-09T15:12:37.807 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:37.801+0000 1694f700 -1 osd.1 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:05.356233+0000 front 2019-12-09T15:11:06.316807+0000 (oldest deadline 2019-12-09T15:11:49.184671+0000) 2019-12-09T15:12:37.808 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:37.803+0000 1694f700 -1 osd.1 269 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:37.841 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:12:37.835+0000 1694f700 -1 osd.4 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:08.860838+0000 front 2019-12-09T15:11:08.809979+0000 (oldest deadline 2019-12-09T15:11:54.058567+0000) 2019-12-09T15:12:37.965 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:12:37.959+0000 1694f700 -1 osd.0 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.630489+0000 front 2019-12-09T15:11:07.672239+0000 (oldest deadline 2019-12-09T15:11:53.520292+0000) 2019-12-09T15:12:38.017 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:12:38.010+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 42sec e269 v269) 2019-12-09T15:12:38.067 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:12:38.062+0000 1694f700 -1 osd.6 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:11:07.805997+0000 front 2019-12-09T15:11:07.909644+0000 (oldest deadline 2019-12-09T15:11:49.459625+0000) 2019-12-09T15:12:38.416 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:38.412+0000 1694f700 -1 osd.5 269 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:11:27.139436+0000 (oldest deadline 2019-12-09T15:12:07.139436+0000) 2019-12-09T15:12:38.417 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:12:38.414+0000 1694f700 -1 osd.5 269 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e267) 2019-12-09T15:12:39.145 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:12:39.140+0000 1694f700 -1 osd.1 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:107690 2.33s0 2:cfcf0c1a:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_bb1bcf0104814a4ea7cd9993b658b140:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e265) 2019-12-09T15:12:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:testIfModifiedSince (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:13:07.261 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:13:07.253+0000 124b0700 -1 osd.2 274 build_incremental_map_msg missing incremental map 274 2019-12-09T15:13:07.322 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:13:07.302+0000 31786700 -1 osd.2 274 build_incremental_map_msg missing incremental map 274 2019-12-09T15:13:07.723 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:13:07.680+0000 37792700 -1 osd.2 274 build_incremental_map_msg missing incremental map 274 2019-12-09T15:13:08.905 INFO:teuthology.orchestra.run.smithi033.stderr:testIfNoneMatch (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:13:40.958 INFO:teuthology.orchestra.run.smithi033.stderr:testIfUnmodifiedSince (test.functional.tests.TestFileComparison) ... ok 2019-12-09T15:13:58.412 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:45:19.871 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.419 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:45:19.877 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.427 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:45:19.886 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.437 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:45:19.897 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.442 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:45:19.923 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.450 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:45:19.932 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.544 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:45:20.003 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.551 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:45:20.011 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:13:58.614 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:45:20.096 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:14:05.757 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:05.752+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:06.717 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:06.712+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:06.803 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:06.798+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:07.440 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:07.435+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:07.709 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:07.704+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:07.742 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:07.737+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:07.801 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:07.796+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:08.437 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:08.431+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:08.738 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:08.731+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:08.769 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:08.762+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:08.813 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:08.807+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:09.408 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:09.403+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:09.732 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:09.726+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:09.749 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:09.742+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:09.841 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:09.835+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:10.291 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:10.287+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:10.363 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:10.359+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:10.542 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:10.537+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:10.706 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:10.698+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:10.753 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:10.746+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:10.831 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:10.825+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:11.250 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:11.246+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:11.419 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:11.414+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:11.510 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:11.506+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:11.669 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:11.662+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:11.775 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:11.769+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:11.789 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:11.778+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:12.257 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:12.253+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:12.441 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:12.438+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:12.496 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:12.493+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:12.699 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:12.685+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:12.766 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:12.760+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:12.776 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:12.770+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:13.244 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:13.240+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:13.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:13.443+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:13.512 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:13.509+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:13.735 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:13.728+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:13.747 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:13.740+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:13.749 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:13.742+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:14.257 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:14.251+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:14.467 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:14.463+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:14.479 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:14.474+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:14.747 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:14.739+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:14.761 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:14.755+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:14.781 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:14.775+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:15.004 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:14.992+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:15.280 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:15.277+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:15.483 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:15.480+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:15.516 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:15.512+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:15.757 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:15.750+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:15.766 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:15.761+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:15.796 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:15.790+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:16.025 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:16.019+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:16.323 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:16.319+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:16.441 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:16.438+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:16.539 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:16.536+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:16.785 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:16.778+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:16.792 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:16.785+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:16.840 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:16.832+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:17.064 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:17.056+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:17.331 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:17.327+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:17.434 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:17.430+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:17.542 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:17.539+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:17.748 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:17.742+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:17.829 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:17.822+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:17.866 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:17.860+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:18.083 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:18.074+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:18.370 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:18.366+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:18.412 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:18.408+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:18.529 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:18.525+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:18.783 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:18.777+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:18.819 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:18.812+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:18.829 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:18.822+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:19.061 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:19.054+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:19.415 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:19.411+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:19.457 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:19.452+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:19.489 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:19.484+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:19.783 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:19.777+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:19.811 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:19.805+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:19.858 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:19.851+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:20.099 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:20.091+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:20.433 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:20.428+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:20.442 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:20.437+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:20.456 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:20.450+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:20.754 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:20.747+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:20.828 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:20.821+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:20.868 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:20.857+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:21.139 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:21.131+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:21.442 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:21.437+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:21.447 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:21.442+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:21.487 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:21.482+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:21.714 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:21.708+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:21.866 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:21.858+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:21.882 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:21.875+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:22.144 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:22.138+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:22.442 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:22.437+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:22.479 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:22.474+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:22.486 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:22.481+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:22.742 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:22.735+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:22.869 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:22.862+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:22.899 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:22.891+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:23.157 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:23.150+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:23.472 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:23.468+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:23.483 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:23.478+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:23.502 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:23.497+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:23.719 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:23.711+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:23.835 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:23.828+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:23.942 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:23.936+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:24.192 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:24.185+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:24.478 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:24.473+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:24.494 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:24.489+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:24.531 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:24.527+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:24.690 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:24.683+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:24.792 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:24.786+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:24.944 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:24.938+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:25.231 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:25.224+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:25.469 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:25.462+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:25.481 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:25.476+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:25.516 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:25.512+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:25.744 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:25.733+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:25.833 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:25.826+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:25.966 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:25.960+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:26.255 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:26.249+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:26.448 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:26.442+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:26.458 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:26.454+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:26.562 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:26.558+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:26.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:26.695+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:26.847 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:26.842+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:27.010 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:27.005+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:27.211 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:27.205+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:27.442 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:27.437+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:27.501 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:27.497+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:27.566 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:27.561+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:27.716 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:27.711+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:27.849 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:27.844+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:28.017 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:28.012+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:28.238 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:28.233+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:28.448 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:28.444+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:28.497 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:28.488+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:28.526 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:28.520+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:28.719 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:28.713+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:28.894 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:28.889+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:28.982 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:28.976+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:29.205 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:29.199+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:29.449 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:29.442+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:29.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:29.490+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:29.511 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:29.508+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:29.726 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:29.720+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:29.888 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:29.882+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:29.944 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:29.938+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:30.232 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:30.227+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:30.461 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:30.457+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:30.507 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:30.502+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:30.528 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:30.524+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:30.713 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:30.708+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:30.913 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:30.906+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:30.957 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:30.951+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:31.271 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:31.264+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:31.465 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:31.461+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:31.528 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:31.523+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:31.556 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:31.551+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:31.680 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:31.674+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:31.868 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:31.861+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:31.922 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:31.915+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:32.255 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:32.240+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:32.468 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:32.462+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:32.519 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:32.514+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:32.573 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:32.567+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:32.659 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:32.653+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:32.872 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:32.867+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:32.958 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:32.953+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:33.275 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:33.269+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:33.495 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:33.490+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:33.518 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:33.512+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:33.579 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:33.575+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:33.715 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:33.708+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:33.904 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:33.898+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:33.950 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:33.944+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:34.244 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:34.237+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:34.484 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:34.479+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:34.547 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:34.543+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:34.628 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:34.623+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:34.714 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:34.707+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:34.945 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:34.940+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:34.966 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:34.961+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:35.201 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:35.194+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:35.492 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:35.485+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:35.514 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:35.510+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:35.626 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:35.621+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:35.767 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:35.756+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:35.987 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:35.981+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:35.989 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:35.982+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:36.213 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:36.206+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:36.485 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:36.480+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:36.542 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:36.537+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:36.587 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:36.581+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:36.819 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:36.814+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:36.951 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:36.946+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:37.027 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:37.021+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:37.189 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:37.184+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:37.503 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:37.497+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:37.562 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:37.558+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:37.593 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:37.589+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:37.808 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:37.801+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:37.990 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:37.984+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:37.991 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:37.986+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:38.153 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:38.144+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:38.374 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:14:38.369+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 42sec e274 v274) 2019-12-09T15:14:38.471 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:38.466+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:38.531 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:38.526+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:38.572 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:38.567+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:38.797 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:38.790+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:39.036 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:39.030+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:39.037 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:39.031+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:39.112 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:39.103+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:39.470 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:39.465+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:39.578 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:39.574+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:39.599 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:39.593+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:39.756 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:39.751+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:40.037 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:40.031+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:40.084 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:40.078+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:40.144 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:40.139+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:40.514 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:40.510+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:40.587 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:40.581+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:40.594 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:40.586+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:40.794 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:40.787+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:41.014 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:41.007+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:41.134 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:41.129+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:41.145 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:41.139+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:41.513 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:41.508+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:41.574 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:41.570+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:41.641 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:41.636+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:41.838 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:41.832+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:41.995 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:41.989+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:42.101 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:42.096+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:42.181 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:42.174+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:42.495 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:42.491+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:42.617 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:42.608+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:42.688 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:42.683+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:42.828 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:42.818+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:42.998 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:42.993+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:43.122 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:43.117+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:43.143 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:43.139+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:43.394 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:14:43.388+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 42sec e274 v274) 2019-12-09T15:14:43.510 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:43.506+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:43.621 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:43.618+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:43.647 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:43.643+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:43.851 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:43.846+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:43.964 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:43.959+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:44.097 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:44.091+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:44.142 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:44.134+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:44.467 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:44.463+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:44.604 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:44.601+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:44.631 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:44.627+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:44.873 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:44.866+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:45.016 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:45.009+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:45.067 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:45.061+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:45.134 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:45.129+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:45.423 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:45.418+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:45.581 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:45.577+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:45.624 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:45.620+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:45.901 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:45.895+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:46.059 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:46.053+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:46.065 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:46.059+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:46.174 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:46.169+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:46.405 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:46.400+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:46.588 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:46.583+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:46.638 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:46.633+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:46.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:46.853+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:47.050 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:47.044+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:47.092 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:47.086+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:47.171 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:47.166+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:47.376 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:47.371+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:47.551 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:47.545+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:14:47.669 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:14:47.664+0000 1694f700 -1 osd.4 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.637116+0000 front 2019-12-09T15:13:24.514464+0000 (oldest deadline 2019-12-09T15:14:07.119203+0000) 2019-12-09T15:14:47.900 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:14:47.895+0000 1694f700 -1 osd.0 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.829844+0000 front 2019-12-09T15:13:23.823907+0000 (oldest deadline 2019-12-09T15:14:06.685013+0000) 2019-12-09T15:14:48.067 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:14:48.061+0000 1694f700 -1 osd.2 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:19.871292+0000 front 2019-12-09T15:13:20.060894+0000 (oldest deadline 2019-12-09T15:14:05.739362+0000) 2019-12-09T15:14:48.084 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:14:48.079+0000 1694f700 -1 osd.1 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.728820+0000 front 2019-12-09T15:13:25.303923+0000 (oldest deadline 2019-12-09T15:14:07.221276+0000) 2019-12-09T15:14:48.209 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:14:48.203+0000 1694f700 -1 osd.3 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:23.227770+0000 front 2019-12-09T15:13:23.228701+0000 (oldest deadline 2019-12-09T15:14:14.041305+0000) 2019-12-09T15:14:48.355 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:14:48.351+0000 1694f700 -1 osd.6 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:24.732437+0000 front 2019-12-09T15:13:25.141141+0000 (oldest deadline 2019-12-09T15:14:10.089893+0000) 2019-12-09T15:14:48.467 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:14:48.461+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 42sec e274 v274) 2019-12-09T15:14:48.579 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:14:48.575+0000 1694f700 -1 osd.7 274 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:13:25.305615+0000 front 2019-12-09T15:13:24.730162+0000 (oldest deadline 2019-12-09T15:14:09.355205+0000) 2019-12-09T15:15:46.236 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:46.230+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:46.832 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:15:46.826+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:02.621887+0000 front 2019-12-09T15:15:02.607303+0000 (oldest deadline 2019-12-09T15:15:46.499169+0000) 2019-12-09T15:15:47.240 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:47.232+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:47.245 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:47.240+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:47.787 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:15:47.782+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:02.621887+0000 front 2019-12-09T15:15:02.607303+0000 (oldest deadline 2019-12-09T15:15:46.499169+0000) 2019-12-09T15:15:48.213 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:48.208+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:48.264 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:48.260+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:48.275 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:48.268+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:48.833 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:15:48.827+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:02.621887+0000 front 2019-12-09T15:15:02.607303+0000 (oldest deadline 2019-12-09T15:15:46.499169+0000) 2019-12-09T15:15:49.186 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:49.180+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:49.187 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:49.181+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:49.250 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:49.244+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:49.309 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:49.303+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:49.366 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:49.360+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:49.540 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:49.535+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:49.868 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:15:49.862+0000 1694f700 -1 osd.3 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:02.621887+0000 front 2019-12-09T15:15:02.607303+0000 (oldest deadline 2019-12-09T15:15:46.499169+0000) 2019-12-09T15:15:50.199 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:50.194+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:50.200 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:50.196+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:50.262 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:50.257+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:50.279 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:50.272+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:50.335 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:50.330+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:50.555 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:50.549+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:51.194 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:51.188+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:51.194 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:51.190+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:51.234 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:51.229+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:51.306 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:51.300+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:51.390 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:51.383+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:51.509 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:51.504+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:52.223 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:52.213+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:52.235 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:52.229+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:52.238 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:52.233+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:52.289 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:52.283+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:52.349 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:52.344+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:52.563 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:52.557+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:53.198 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:53.190+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:53.199 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:53.193+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:53.262 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:53.254+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:53.303 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:53.293+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:53.387 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:53.379+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:53.599 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:53.593+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:54.185 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:54.180+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:54.186 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:54.182+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:54.259 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:54.254+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:54.339 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:54.333+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:54.356 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:54.351+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:54.592 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:54.586+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:55.176 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:55.169+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:55.177 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:55.172+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:55.245 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:55.240+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:55.362 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:55.357+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:55.393 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:55.388+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:55.558 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:55.553+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:56.169 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:56.163+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:56.170 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:56.165+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:56.245 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:56.240+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:56.373 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:56.368+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:56.383 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:56.377+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:56.607 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:56.603+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:57.136 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:57.130+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:57.137 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:57.132+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:57.239 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:57.233+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:57.341 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:57.335+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:57.368 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:57.362+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:57.618 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:57.612+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:58.129 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:58.123+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:58.132 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:58.128+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:58.193 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:58.188+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:58.388 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:58.383+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:58.422 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:58.416+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:58.573 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:15:58.569+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:15:58.659 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:58.654+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:15:59.138 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:59.125+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:15:59.138 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:15:59.127+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:15:59.235 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:15:59.228+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:15:59.410 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:15:59.405+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:15:59.456 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:15:59.448+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:15:59.530 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:15:59.527+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:15:59.625 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:15:59.621+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:00.111 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:00.104+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:00.113 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:00.109+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:00.221 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:00.215+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:00.466 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:00.460+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:00.514 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:00.510+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:00.515 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:00.511+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:00.658 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:00.655+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:01.116 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:01.108+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:01.116 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:01.112+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:01.241 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:01.235+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:01.452 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:01.445+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:01.497 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:01.494+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:01.531 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:01.526+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:01.713 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:01.708+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:02.134 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:02.128+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:02.136 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:02.131+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:02.242 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:02.236+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:02.468 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:02.463+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:02.532 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:02.527+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:02.533 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:02.529+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:02.698 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:02.693+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:03.134 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:03.126+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:03.135 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:03.130+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:03.242 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:03.237+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:03.430 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:03.426+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:03.491 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:03.488+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:03.548 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:03.542+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:03.677 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:03.671+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:04.164 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:04.156+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:04.164 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:04.159+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:04.288 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:04.282+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:04.443 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:04.438+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:04.503 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:04.498+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:04.517 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:04.509+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:04.710 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:04.706+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:05.176 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:05.169+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:05.176 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:05.172+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:05.262 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:05.256+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:05.417 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:05.413+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:05.546 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:05.541+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:05.568 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:05.563+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:05.741 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:05.736+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:06.149 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:06.143+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:06.150 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:06.145+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:06.241 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:06.234+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:06.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:06.439+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:06.582 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:06.575+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:06.595 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:06.590+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:06.712 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:06.706+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:07.121 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:07.114+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:07.122 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:07.117+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:07.228 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:07.222+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:07.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:07.440+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:07.592 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:07.587+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:07.632 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:07.626+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:07.744 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:07.736+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:08.138 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:08.131+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:08.139 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:08.133+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:08.238 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:08.231+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:08.424 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:08.418+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:08.630 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:08.624+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:08.646 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:08.640+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:08.706 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:08.697+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:09.121 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:09.111+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:09.122 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:09.113+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:09.271 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:09.264+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:09.416 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:09.410+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:09.611 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:09.607+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:09.660 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:09.654+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:09.702 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:09.696+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:10.089 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:10.081+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:10.090 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:10.083+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:10.261 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:10.253+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:10.462 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:10.456+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:10.630 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:10.625+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:10.655 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:10.649+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:10.671 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:10.665+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:11.114 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:11.107+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:11.115 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:11.108+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:11.233 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:11.226+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:11.480 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:11.475+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:11.639 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:11.635+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:11.686 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:11.681+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:11.714 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:11.709+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:12.105 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:12.097+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:12.106 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:12.099+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:12.198 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:12.190+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:12.466 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:12.461+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:12.650 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:12.645+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:12.691 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:12.685+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:12.727 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:12.721+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:13.109 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:13.096+0000 1694f700 -1 osd.1 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:01.512124+0000 front 2019-12-09T15:15:01.359135+0000 (oldest deadline 2019-12-09T15:15:46.037376+0000) 2019-12-09T15:16:13.109 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:13.098+0000 1694f700 -1 osd.1 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:13.244 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:13.234+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:13.435 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:16:13.429+0000 1694f700 -1 osd.4 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:04.659200+0000 front 2019-12-09T15:15:04.681376+0000 (oldest deadline 2019-12-09T15:15:46.318127+0000) 2019-12-09T15:16:13.661 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:16:13.656+0000 1694f700 -1 osd.5 280 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e278) 2019-12-09T15:16:13.704 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:16:13.699+0000 1694f700 -1 osd.6 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.093905+0000 front 2019-12-09T15:15:03.030722+0000 (oldest deadline 2019-12-09T15:15:48.852010+0000) 2019-12-09T15:16:13.777 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:16:13.771+0000 1694f700 -1 osd.7 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:03.422283+0000 front 2019-12-09T15:15:03.424921+0000 (oldest deadline 2019-12-09T15:15:48.466361+0000) 2019-12-09T15:16:14.126 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:14.104+0000 1694f700 -1 osd.1 281 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:16:14.198 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:16:14.191+0000 1694f700 -1 osd.0 280 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:15:05.268469+0000 front 2019-12-09T15:15:05.235655+0000 (oldest deadline 2019-12-09T15:15:47.506388+0000) 2019-12-09T15:16:15.313 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:16:15.123+0000 1694f700 -1 osd.1 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:108330 2.33s0 2:cf3c3b40:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.656_cfd5c865c91f49c7b0d628b734643386:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e276) 2019-12-09T15:17:03.132 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:17:03.119+0000 31786700 -1 osd.2 286 build_incremental_map_msg missing incremental map 286 2019-12-09T15:17:03.169 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:17:03.144+0000 35f8f700 -1 osd.2 286 build_incremental_map_msg missing incremental map 286 2019-12-09T15:17:03.405 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:17:03.299+0000 3478c700 -1 osd.2 286 build_incremental_map_msg missing incremental map 286 2019-12-09T15:17:49.254 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:49:10.713 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.263 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:49:10.722 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.268 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:49:10.728 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.274 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:49:10.733 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.276 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:49:10.757 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.281 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:49:10.762 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.509 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:49:10.968 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.520 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:49:10.979 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:49.524 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:49:11.006 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:17:58.777 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:17:58.771+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:17:59.593 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:17:59.587+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:17:59.785 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:17:59.778+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:00.307 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:00.303+0000 36790700 -1 osd.2 286 build_incremental_map_msg missing incremental map 251 2019-12-09T15:18:00.327 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:00.323+0000 36790700 -1 osd.2 286 build_incremental_map_msg missing incremental map 274 2019-12-09T15:18:00.344 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:00.333+0000 36790700 -1 osd.2 286 build_incremental_map_msg missing incremental map 286 2019-12-09T15:18:00.346 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:00.341+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:00.592 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:00.585+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:00.628 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:00.621+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:00.806 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:00.799+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:01.040 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:01.035+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:01.139 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:01.132+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:01.394 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:01.386+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:01.556 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:01.549+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:01.584 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:01.577+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:01.622 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:01.618+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:01.797 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:01.792+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:02.063 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:02.058+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:02.152 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:02.147+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:02.443 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:02.438+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:02.542 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:02.536+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:02.630 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:02.623+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:02.649 INFO:teuthology.orchestra.run.smithi033.stderr:testIfMatch (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:18:02.657 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:02.652+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:02.796 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:02.791+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:03.056 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:03.051+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:03.155 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:03.148+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:03.490 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:03.486+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:03.525 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:03.519+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:03.642 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:03.637+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:03.647 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:03.641+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:03.847 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:03.841+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:04.033 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:04.029+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:04.175 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:04.170+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:04.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:04.440+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:04.499 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:04.494+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:04.603 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:04.598+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:04.690 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:04.684+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:04.845 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:04.839+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:04.998 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:04.994+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:05.165 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:05.159+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:05.463 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:05.458+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:05.507 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:05.502+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:05.594 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:05.589+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:05.684 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:05.678+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:05.870 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:05.864+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:06.042 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:06.038+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:06.185 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:06.178+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:06.468 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:06.464+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:06.516 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:06.511+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:06.623 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:06.618+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:06.720 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:06.714+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:06.828 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:06.822+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:07.020 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:07.017+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:07.232 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:07.227+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:07.505 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:07.501+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:07.513 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:07.508+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:07.674 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:07.668+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:07.742 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:07.737+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:07.794 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:07.788+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:07.989 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:07.984+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:08.231 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:08.226+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:08.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:08.488+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:08.507 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:08.501+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:08.630 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:08.626+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:08.704 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:08.698+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:08.835 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:08.827+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:08.977 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:08.972+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:09.218 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:09.211+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:09.507 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:09.502+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:09.515 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:09.507+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:09.656 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:09.651+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:09.703 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:09.697+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:09.870 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:09.847+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:09.982 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:09.977+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:10.248 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:10.243+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:10.465 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:10.460+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:10.554 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:10.549+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:10.703 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:10.698+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:10.731 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:10.725+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:10.833 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:10.826+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:11.020 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:11.014+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:11.256 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:11.251+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:11.431 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:11.426+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:11.546 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:11.540+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:11.682 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:11.677+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:11.722 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:11.716+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:11.786 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:11.780+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:12.018 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:12.014+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:12.288 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:12.282+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:12.407 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:12.401+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:12.501 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:12.495+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:12.703 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:12.698+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:12.723 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:12.716+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:12.828 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:12.822+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:13.063 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:13.059+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:13.313 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:13.306+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:13.460 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:13.455+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:13.505 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:13.499+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:13.710 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:13.703+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:13.731 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:13.726+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:13.841 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:13.835+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:14.035 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:14.030+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:14.306 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:14.298+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:14.471 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:14.467+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:14.545 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:14.535+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:14.709 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:14.702+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:14.743 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:14.738+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:14.835 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:14.829+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:15.029 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:15.025+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:15.273 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:15.267+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:15.461 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:15.456+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:15.535 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:15.528+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:15.730 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:15.724+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:15.795 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:15.790+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:15.799 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:15.789+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:16.056 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:16.050+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:16.281 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:16.274+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:16.508 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:16.502+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:16.514 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:16.510+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:16.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:16.737+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:16.792 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:16.784+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:16.829 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:16.824+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:17.066 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:17.061+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:17.273 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:17.260+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:17.470 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:17.465+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:17.548 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:17.541+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:17.744 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:17.736+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:17.820 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:17.813+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:17.851 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:17.846+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:18.041 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:18.036+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:18.274 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:18.266+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:18.497 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:18.491+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:18.589 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:18.582+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:18.712 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:18.706+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:18.794 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:18.787+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:18.877 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:18.872+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:19.051 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:19.046+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:19.245 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:19.239+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:19.538 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:19.532+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:19.633 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:19.625+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:19.741 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:19.734+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:19.799 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:19.791+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:19.921 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:19.916+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:20.092 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:20.087+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:20.249 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:20.244+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:20.514 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:20.505+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:20.604 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:20.595+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:20.714 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:20.707+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:20.835 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:20.827+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:20.918 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:20.913+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:21.079 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:21.075+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:21.248 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:21.241+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:21.464 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:21.460+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:21.630 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:21.617+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:21.729 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:21.722+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:21.879 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:21.874+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:21.928 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:21.924+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:22.041 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:22.036+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:22.230 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:22.224+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:22.500 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:22.495+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:22.649 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:22.642+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:22.685 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:22.679+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:22.856 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:22.850+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:22.935 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:22.931+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:23.081 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:23.077+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:23.192 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:23.185+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:23.475 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:23.471+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:23.632 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:23.626+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:23.727 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:23.721+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:23.861 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:23.855+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:23.936 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:23.930+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:24.117 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:24.111+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:24.166 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:24.159+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:24.489 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:24.484+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:24.657 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:24.650+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:24.753 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:24.747+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:24.818 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:24.811+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:24.901 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:24.896+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:25.134 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:25.129+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:25.214 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:25.207+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:25.487 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:25.483+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:25.684 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:25.678+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:25.766 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:25.758+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:25.843 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:25.831+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:25.890 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:25.885+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:26.170 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:26.164+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:26.236 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:26.229+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:26.491 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:26.485+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:26.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:26.667+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:26.798 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:26.792+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:26.825 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:26.818+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:26.933 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:26.928+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:27.175 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:27.172+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:27.195 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:27.188+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:27.536 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:27.531+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:27.655 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:27.648+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:27.781 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:27.774+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:27.827 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:27.821+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:27.889 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:27.884+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:28.150 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:28.143+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:28.210 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:28.205+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:28.548 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:28.543+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:28.650 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:28.644+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:28.764 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:28.757+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:28.859 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:28.851+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:28.873 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:28.868+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:29.179 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:29.174+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:29.202 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:29.195+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:29.584 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:29.578+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:29.659 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:29.650+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:29.803 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:29.797+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:29.909 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:29.904+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:29.909 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:29.904+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:30.217 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:30.211+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:30.252 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:30.246+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:30.555 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:30.550+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:30.674 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:30.668+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:30.784 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:30.779+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:30.881 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:30.877+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:30.960 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:30.954+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:31.195 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:31.189+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:31.285 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:31.280+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:31.520 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:31.515+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:31.678 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:31.673+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:31.827 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:31.821+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:31.839 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:31.834+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:31.923 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:31.918+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:32.203 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:32.198+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:32.273 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:32.266+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:32.489 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:32.483+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:32.672 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:32.666+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:32.795 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:32.789+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:32.808 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:32.802+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:32.897 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:32.891+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:33.202 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:33.198+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:33.253 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:33.244+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:33.476 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:33.470+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:33.704 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:33.698+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:33.767 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:33.763+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:33.799 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:33.793+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:33.852 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:33.847+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:34.161 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:34.156+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:34.202 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:18:34.190+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e286 v286) 2019-12-09T15:18:34.254 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:34.248+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:34.497 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:34.492+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:34.757 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:34.751+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:34.766 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:34.761+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:34.832 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:34.826+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:34.833 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:34.826+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:35.193 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:35.189+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:35.267 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:35.262+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:35.501 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:35.496+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:35.735 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:35.730+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:35.743 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:35.738+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:35.842 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:35.836+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:35.865 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:35.859+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:36.183 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:36.177+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:36.322 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:36.316+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:36.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:36.489+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:36.688 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:36.683+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:36.699 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:36.694+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:36.885 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:36.878+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:36.907 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:36.902+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:37.163 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:37.159+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:37.285 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:37.279+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:37.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:37.488+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:37.652 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:37.646+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:37.750 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:37.744+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:37.874 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:37.869+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:37.901 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:37.896+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:38.137 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:38.133+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:38.244 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:38.237+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:18:38.533 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:18:38.529+0000 1694f700 -1 osd.4 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.548253+0000 front 2019-12-09T15:17:15.428334+0000 (oldest deadline 2019-12-09T15:17:59.480278+0000) 2019-12-09T15:18:38.613 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:18:38.608+0000 1694f700 -1 osd.6 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.006369+0000 front 2019-12-09T15:17:18.780224+0000 (oldest deadline 2019-12-09T15:18:01.194898+0000) 2019-12-09T15:18:38.751 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:18:38.743+0000 1694f700 -1 osd.0 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.311015+0000 front 2019-12-09T15:17:19.539559+0000 (oldest deadline 2019-12-09T15:18:00.038530+0000) 2019-12-09T15:18:38.902 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:18:38.898+0000 1694f700 -1 osd.1 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:13.597418+0000 front 2019-12-09T15:17:13.834460+0000 (oldest deadline 2019-12-09T15:17:59.417561+0000) 2019-12-09T15:18:38.910 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:18:38.905+0000 1694f700 -1 osd.3 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.787120+0000 front 2019-12-09T15:17:18.931496+0000 (oldest deadline 2019-12-09T15:17:58.677927+0000) 2019-12-09T15:18:39.181 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:18:39.177+0000 1694f700 -1 osd.7 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:17.655624+0000 front 2019-12-09T15:17:19.548905+0000 (oldest deadline 2019-12-09T15:18:00.516930+0000) 2019-12-09T15:18:39.246 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:18:39.240+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 43sec e286 v286) 2019-12-09T15:18:39.287 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:18:39.282+0000 1694f700 -1 osd.2 286 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:17:19.583830+0000 front 2019-12-09T15:17:15.674580+0000 (oldest deadline 2019-12-09T15:18:00.242505+0000) 2019-12-09T15:19:16.922 INFO:teuthology.orchestra.run.smithi033.stderr:testIfMatchAndUnmodified (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:19:35.987 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:35.981+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:37.024 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:37.020+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:37.851 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:37.845+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:37.986 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:37.982+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:38.078 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:38.073+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:38.208 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:38.202+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:38.825 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:38.819+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:38.998 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:38.993+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:39.113 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:39.108+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:39.228 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:39.224+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:39.796 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:39.790+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:40.006 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:40.002+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:40.097 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:40.091+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:40.112 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:40.107+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:40.193 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:40.186+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:40.848 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:40.842+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:41.011 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:41.006+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:41.086 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:41.079+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:41.092 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:41.086+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:41.214 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:41.207+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:41.335 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:41.329+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:41.861 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:41.855+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:41.975 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:41.970+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:42.055 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:42.049+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:42.078 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:42.073+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:42.232 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:42.225+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:42.357 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:42.352+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:42.825 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:42.820+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:42.995 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:42.989+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:43.058 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:43.048+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:43.125 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:43.120+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:43.210 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:43.205+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:43.342 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:43.336+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:43.843 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:43.836+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:44.036 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:44.030+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:44.087 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:44.081+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:44.092 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:44.086+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:44.241 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:44.233+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:44.358 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:44.353+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:44.870 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:44.864+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:45.080 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:45.074+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:45.083 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:45.077+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:45.133 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:45.128+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:45.267 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:45.262+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:45.368 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:45.361+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:45.841 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:45.834+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:46.073 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:46.068+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:46.123 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:46.115+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:46.142 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:46.136+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:46.283 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:46.278+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:46.339 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:46.334+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:46.857 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:46.851+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:47.109 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:47.103+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:47.111 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:47.106+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:47.175 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:47.168+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:47.259 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:47.251+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:47.358 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:47.353+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:47.812 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:47.806+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:48.070 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:48.063+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:48.078 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:48.072+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:48.204 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:48.196+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:48.292 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:48.287+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:48.417 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:48.405+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:48.850 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:48.842+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:49.030 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:49.025+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:49.166 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:49.137+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:49.186 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:49.181+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:49.252 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:49.248+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:49.459 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:49.433+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:49.826 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:49.821+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:50.023 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:50.018+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:50.145 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:50.138+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:50.177 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:50.170+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:50.217 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:50.211+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:50.478 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:50.472+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:50.874 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:50.868+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:51.065 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:51.060+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:51.193 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:51.187+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:51.229 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:51.224+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:51.234 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:51.228+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:51.481 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:51.474+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:51.886 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:51.881+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:52.114 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:52.109+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:52.155 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:52.149+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:52.265 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:52.258+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:52.272 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:52.266+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:52.502 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:52.496+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:52.858 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:52.852+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:53.114 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:53.109+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:53.126 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:53.120+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:53.289 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:53.284+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:53.302 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:53.296+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:53.448 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:53.443+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:19:53.517 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:53.512+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:53.872 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:53.867+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:54.081 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:54.076+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:54.112 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:54.106+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:54.255 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:54.249+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:54.339 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:54.334+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:54.490 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:54.485+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:54.493 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:54.488+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:19:54.868 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:54.862+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:55.112 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:55.106+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:55.124 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:55.118+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:55.305 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:55.300+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:55.352 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:55.347+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:55.503 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:55.499+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:19:55.513 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:55.505+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:55.890 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:55.884+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:56.104 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:56.097+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:56.121 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:56.115+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:56.282 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:56.275+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:56.362 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:56.356+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:56.472 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:56.467+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:56.547 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:56.543+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:19:56.931 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:56.925+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:57.113 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:57.108+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:57.135 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:57.130+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:57.307 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:57.302+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:57.393 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:57.387+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:57.495 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:57.490+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:57.596 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:57.589+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:19:57.598 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:57.593+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:19:57.970 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:57.962+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:58.154 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:58.149+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:58.179 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:58.172+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:58.335 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:58.329+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:58.377 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:58.371+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:58.523 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:58.518+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:58.559 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:58.555+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:19:58.562 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:58.558+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:19:58.987 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:19:58.982+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:19:59.199 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:19:59.194+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:19:59.203 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:19:59.198+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:19:59.335 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:19:59.329+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:19:59.382 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:19:59.376+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:19:59.489 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:19:59.482+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:19:59.540 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:59.533+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:19:59.541 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:19:59.536+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:00.013 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:00.007+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:00.232 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:00.226+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:00.244 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:00.238+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:00.308 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:00.302+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:00.354 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:00.348+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:00.481 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:00.451+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:00.545 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:00.538+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:00.549 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:00.544+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:00.576 INFO:teuthology.orchestra.run.smithi033.stderr:testIfModifiedSince (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:20:01.056 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:01.051+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:01.270 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:01.264+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:01.297 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:01.291+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:01.324 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:01.318+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:01.338 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:01.333+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:01.548 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:01.543+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:01.565 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:01.560+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:01.569 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:01.565+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:02.077 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:02.071+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:02.297 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:02.287+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:02.324 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:02.319+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:02.369 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:02.363+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:02.375 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:02.369+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:02.522 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:02.516+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:02.605 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:02.597+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:02.607 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:02.601+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:03.065 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:03.058+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:03.308 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:03.302+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:03.327 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:03.321+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:03.332 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:03.327+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:03.413 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:03.407+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:03.586 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:03.581+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:03.602 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:03.594+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:03.604 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:03.599+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:04.065 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:04.059+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:04.298 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:04.291+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:04.310 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:04.305+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:04.347 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:04.341+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:04.433 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:04.425+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:04.565 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:04.559+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:04.566 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:04.561+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:04.618 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:04.604+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:05.066 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:05.061+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:05.310 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:05.304+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:05.328 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:05.322+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:05.339 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:05.332+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:05.483 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:05.477+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:05.529 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:05.522+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:05.534 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:05.528+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:05.626 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:05.616+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:06.047 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:06.041+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:06.280 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:06.274+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:06.301 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:06.295+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:06.326 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:06.320+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:06.492 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:06.486+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:06.586 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:06.577+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:06.587 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:06.579+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:06.624 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:06.617+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:07.070 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:07.058+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:07.306 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:07.300+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:07.311 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:07.305+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:07.330 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:07.325+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:07.527 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:07.522+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:07.540 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:07.534+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:07.542 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:07.537+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:07.647 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:07.631+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:08.107 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:08.101+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:08.322 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:08.315+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:08.344 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:08.339+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:08.350 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:08.345+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:08.491 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:08.483+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:08.560 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:08.553+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:08.562 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:08.556+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:08.678 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:08.673+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:09.086 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:09.081+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:09.288 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:09.283+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:09.347 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:09.341+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:09.398 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:09.392+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:09.465 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:09.458+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:09.474 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:20:09.470+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e293 v293) 2019-12-09T15:20:09.572 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:09.567+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:09.573 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:09.569+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:09.699 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:09.693+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:10.118 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:10.113+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:10.282 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:10.273+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:10.336 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:10.329+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:10.387 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:10.381+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:10.469 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:10.463+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:10.571 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:10.547+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:10.606 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:10.600+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:10.734 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:10.729+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:11.135 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:11.128+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:11.280 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:11.272+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:11.310 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:11.304+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:11.444 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:11.438+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:11.475 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:11.469+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:11.625 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:11.618+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:11.627 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:11.621+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:11.781 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:11.775+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:12.113 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:12.107+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:12.234 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:12.229+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:12.330 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:12.323+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:12.440 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:12.434+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:12.483 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:12.477+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:12.595 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:12.588+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:12.597 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:12.591+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:12.832 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:12.825+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:13.165 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:13.159+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:13.203 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:13.196+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:13.312 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:13.307+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:13.466 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:13.460+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:13.478 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:13.473+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:13.626 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:13.620+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:13.627 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:13.623+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:13.788 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:13.781+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:14.169 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:14.162+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:14.208 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:14.203+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:14.284 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:14.278+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:14.483 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:14.477+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:14.484 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:20:14.479+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e293 v293) 2019-12-09T15:20:14.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:14.489+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:14.599 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:14.593+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:14.599 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:14.595+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:14.749 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:14.743+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:15.169 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:15.163+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:15.210 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:15.206+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:15.336 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:15.331+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:15.451 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:15.444+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:15.471 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:15.466+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:15.613 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:15.608+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:15.617 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:15.613+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:15.746 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:15.740+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:16.158 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:16.150+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:16.189 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:16.182+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:16.366 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:16.359+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:16.483 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:16.477+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:16.483 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:16.477+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:16.622 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:16.615+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:16.622 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:16.618+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:16.711 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:16.704+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:17.169 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:17.155+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:17.197 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:17.190+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:17.385 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:17.378+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:17.480 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:17.475+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:17.529 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:17.523+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:17.639 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:17.633+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:17.640 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:17.636+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:17.732 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:17.725+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:18.179 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:18.173+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:18.183 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:18.176+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:18.423 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:18.416+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:18.494 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:18.488+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:18.559 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:18.553+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:18.652 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:18.644+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:18.653 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:18.648+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:18.745 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:18.738+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:19.138 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:19.133+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:19.212 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:19.205+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:19.477 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:19.470+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:19.497 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:19.491+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:19.501 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:20:19.496+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e293 v293) 2019-12-09T15:20:19.609 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:19.603+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:19.705 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:19.699+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:19.706 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:19.702+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:19.763 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:19.756+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:20.157 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:20.146+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:20.181 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:20.173+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:20.437 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:20.430+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:20.536 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:20.530+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:20.620 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:20.614+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:20.703 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:20.697+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:20.708 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:20.703+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:20.770 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:20.762+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:21.136 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:21.131+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:21.212 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:21.205+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:21.402 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:21.395+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:21.587 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:21.582+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:21.588 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:21.581+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:21.760 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:21.755+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:21.765 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:21.760+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:21.803 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:21.783+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:22.147 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:22.142+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:22.206 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:22.198+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:22.405 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:22.398+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:22.549 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:22.544+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:22.629 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:22.623+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:22.788 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:22.777+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:22.792 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:22.785+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:22.792 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:22.787+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:23.200 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:23.195+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:23.214 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:23.207+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:23.457 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:23.451+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:23.550 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:23.545+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:23.656 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:23.650+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:23.754 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:23.746+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:23.755 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:23.750+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:23.801 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:23.794+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:23.885 INFO:teuthology.orchestra.run.smithi033.stderr:testIfNoneMatch (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:20:24.193 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:24.188+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:24.227 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:24.218+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:24.450 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:24.443+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:24.529 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:20:24.520+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e293 v293) 2019-12-09T15:20:24.584 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:24.578+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:24.703 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:24.698+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:24.726 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:24.721+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:24.726 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:24.722+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:24.864 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:24.854+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:25.184 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:25.177+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:25.207 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:25.201+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:25.425 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:25.418+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:25.541 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:25.537+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:25.712 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:25.707+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:25.725 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:25.720+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:25.726 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:25.722+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:25.919 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:25.910+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:26.178 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:26.171+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:26.229 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:26.222+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:26.416 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:26.409+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:26.554 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:26.548+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:26.733 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:26.728+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:26.736 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:26.730+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:26.745 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:26.735+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:26.953 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:26.947+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:27.144 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:27.138+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:27.276 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:27.272+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:27.422 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:27.414+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:27.594 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:27.589+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:27.712 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:27.706+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:27.753 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:27.746+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:27.754 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:27.749+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:27.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:27.941+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:28.121 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:28.115+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:28.270 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:28.265+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:28.446 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:28.439+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:28.641 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:28.636+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:28.712 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:28.706+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:28.737 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:28.732+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:28.741 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:28.737+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:28.910 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:28.903+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:29.081 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:29.074+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:29.266 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:29.261+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:29.503 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:29.494+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:29.543 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:20:29.536+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e293 v293) 2019-12-09T15:20:29.638 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:29.632+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:29.720 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:29.714+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:29.720 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:29.714+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:29.720 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:29.716+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:29.884 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:29.877+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:30.056 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:30.047+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:30.226 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:30.222+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:30.513 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:30.500+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:30.594 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:30.590+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:30.698 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:30.694+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:30.754 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:30.749+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:30.755 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:30.751+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:30.915 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:30.907+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:31.042 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:31.035+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:31.270 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:31.265+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:31.544 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:31.537+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:31.644 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:31.639+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:31.725 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:31.719+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:31.792 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:31.788+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:31.796 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:31.793+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:31.938 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:31.929+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:32.092 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:32.084+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:32.246 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:32.241+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:32.570 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:32.563+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:32.685 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:32.680+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:32.734 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:32.726+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:32.812 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:32.808+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:32.816 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:32.813+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:32.937 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:32.931+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:33.134 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:33.128+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:33.245 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:33.241+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:33.545 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:33.535+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:33.646 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:33.642+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:33.725 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:33.721+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:33.857 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:33.852+0000 1694f700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 ever on either front or back, first ping sent 2019-12-09T15:19:16.704170+0000 (oldest deadline 2019-12-09T15:19:56.704170+0000) 2019-12-09T15:20:33.860 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:33.857+0000 1694f700 -1 osd.5 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:33.935 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:20:33.928+0000 1694f700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.769689+0000 front 2019-12-09T15:18:55.770477+0000 (oldest deadline 2019-12-09T15:19:40.964266+0000) 2019-12-09T15:20:34.109 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:20:34.101+0000 1694f700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:53.556303+0000 front 2019-12-09T15:18:53.604646+0000 (oldest deadline 2019-12-09T15:19:37.047801+0000) 2019-12-09T15:20:34.253 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:20:34.248+0000 1694f700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.867698+0000 front 2019-12-09T15:18:52.170795+0000 (oldest deadline 2019-12-09T15:19:37.156873+0000) 2019-12-09T15:20:34.530 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:20:34.525+0000 1694f700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:55.118535+0000 front 2019-12-09T15:18:55.135365+0000 (oldest deadline 2019-12-09T15:19:39.128137+0000) 2019-12-09T15:20:34.609 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:20:34.595+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e293 v293) 2019-12-09T15:20:34.624 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:20:34.619+0000 1694f700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:52.457059+0000 front 2019-12-09T15:18:52.580670+0000 (oldest deadline 2019-12-09T15:19:37.727049+0000) 2019-12-09T15:20:34.721 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:20:34.717+0000 1694f700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:18:51.866049+0000 front 2019-12-09T15:18:56.046426+0000 (oldest deadline 2019-12-09T15:19:35.911209+0000) 2019-12-09T15:20:34.920 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:20:34.854+0000 1694f700 -1 osd.5 294 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4370.0:82306 7.3 7.6605b063 (undecoded) ondisk+retry+read+known_if_redirected e290) 2019-12-09T15:20:49.806 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:52:11.265 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:49.852 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:52:11.312 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:49.854 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:52:11.314 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:49.856 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:52:11.316 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:49.880 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:52:11.361 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:49.901 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:52:11.375 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:50.029 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:52:11.488 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:50.086 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:52:11.546 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:50.204 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:52:11.685 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:55.575 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:20:55.531+0000 31f87700 -1 osd.2 298 build_incremental_map_msg missing incremental map 298 2019-12-09T15:20:55.588 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:20:55.559+0000 35f8f700 -1 osd.2 298 build_incremental_map_msg missing incremental map 298 2019-12-09T15:20:55.640 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:20:55.604+0000 3378a700 -1 osd.2 298 build_incremental_map_msg missing incremental map 298 2019-12-09T15:20:55.717 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:20:55.708+0000 38794700 -1 osd.2 298 build_incremental_map_msg missing incremental map 298 2019-12-09T15:21:24.835 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:21:24.829+0000 32788700 -1 osd.2 299 build_incremental_map_msg missing incremental map 298 2019-12-09T15:21:25.971 INFO:teuthology.orchestra.run.smithi033.stderr:testIfUnmodifiedSince (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-12-09T15:21:48.075 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:53:09.534 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.079 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:53:09.539 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.093 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:53:09.575 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.094 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:53:09.553 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.101 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:53:09.583 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.101 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:53:09.561 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.179 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:53:09.639 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.189 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:53:09.648 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:48.196 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:53:09.678 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:21:53.930 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:21:53.925+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:21:54.028 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:21:54.022+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:21:54.336 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:21:54.331+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:21:54.680 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:21:54.675+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:21:54.848 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:21:54.842+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:21:54.893 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:21:54.885+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:21:54.991 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:21:54.985+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:21:55.358 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:21:55.354+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:21:55.377 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:21:55.373+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:21:55.600 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:21:55.595+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:21:55.724 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:21:55.717+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:21:55.871 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:21:55.864+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:21:55.877 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:21:55.871+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:21:55.982 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:21:55.976+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:21:56.364 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:21:56.359+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:21:56.384 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:21:56.380+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:21:56.579 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:21:56.574+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:21:56.700 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:21:56.694+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:21:56.845 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:21:56.839+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:21:56.878 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:21:56.872+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:21:56.981 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:21:56.969+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:21:57.346 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:21:57.338+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:21:57.399 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:21:57.394+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:21:57.582 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:21:57.576+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:21:57.661 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:21:57.655+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:21:57.872 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:21:57.866+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:21:57.892 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:21:57.885+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:21:57.947 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:21:57.943+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:21:58.316 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:21:58.312+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:21:58.385 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:21:58.381+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:21:58.606 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:21:58.600+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:21:58.623 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:21:58.617+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:21:58.880 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:21:58.874+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:21:58.886 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:21:58.881+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:21:58.948 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:21:58.944+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:21:59.283 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:21:59.278+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:21:59.355 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:21:59.351+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:21:59.581 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:21:59.576+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:21:59.658 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:21:59.653+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:21:59.904 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:21:59.897+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:21:59.930 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:21:59.925+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:21:59.938 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:21:59.932+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:00.319 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:00.314+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:00.331 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:00.327+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:00.580 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:00.575+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:00.636 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:00.629+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:00.890 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:00.885+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:00.891 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:00.887+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:00.924 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:00.919+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:01.357 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:01.353+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:01.371 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:01.366+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:01.557 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:01.548+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:01.635 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:01.630+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:01.864 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:01.857+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:01.914 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:01.910+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:01.936 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:01.931+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:02.333 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:02.328+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:02.362 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:02.356+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:02.578 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:02.562+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:02.655 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:02.650+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:02.870 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:02.865+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:02.891 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:02.886+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:02.934 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:02.928+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:03.336 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:03.331+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:03.371 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:03.365+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:03.560 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:03.553+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:03.649 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:03.643+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:03.846 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:03.841+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:03.847 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:03.843+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:03.893 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:03.888+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:04.362 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:04.358+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:04.375 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:04.370+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:04.543 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:04.537+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:04.618 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:04.612+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:04.884 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:04.878+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:04.893 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:04.887+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:04.932 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:04.927+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:05.344 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:05.339+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:05.352 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:05.347+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:05.574 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:05.568+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:05.627 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:05.621+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:05.857 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:05.852+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:05.905 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:05.900+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:05.941 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:05.936+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:06.309 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:06.305+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:06.400 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:06.394+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:06.532 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:06.524+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:06.592 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:06.587+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:06.849 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:06.843+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:06.955 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:06.949+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:06.959 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:06.954+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:07.356 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:07.349+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:07.393 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:07.388+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:07.512 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:07.507+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:07.592 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:07.586+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:07.818 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:07.813+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:07.916 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:07.911+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:07.922 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:07.916+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:08.386 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:08.382+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:08.427 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:08.423+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:08.548 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:08.542+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:08.553 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:08.547+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:08.862 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:08.856+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:08.916 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:08.910+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:08.974 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:08.968+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:09.388 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:09.383+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:09.406 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:09.400+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:09.506 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:09.501+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:09.573 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:09.567+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:09.819 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:09.814+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:09.953 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:09.947+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:09.986 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:09.980+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:10.391 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:10.386+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:10.411 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:10.408+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:10.551 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:10.545+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:10.564 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:10.558+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:10.786 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:10.779+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:10.964 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:10.957+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:10.965 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:10.960+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:11.436 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:11.431+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:11.458 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:11.453+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:11.531 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:11.525+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:11.536 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:11.530+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:11.811 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:11.805+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:11.931 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:11.921+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:11.965 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:11.958+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:12.398 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:12.393+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:12.496 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:12.492+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:12.535 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:12.529+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:12.539 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:12.533+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:12.851 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:12.846+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:12.899 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:12.895+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:12.925 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:12.918+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:13.401 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:13.395+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:13.497 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:13.493+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:13.544 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:13.538+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:13.589 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:13.582+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:13.877 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:13.870+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:13.888 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:13.879+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:13.903 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:13.898+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:14.430 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:14.425+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:14.496 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:14.490+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:14.548 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:14.542+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:14.620 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:14.614+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:14.887 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:14.882+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:14.890 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:14.884+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:14.908 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:14.902+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:15.420 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:15.415+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:15.467 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:15.463+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:15.545 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:15.539+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:15.644 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:15.639+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:15.908 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:15.903+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:15.932 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:15.920+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:15.943 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:15.936+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:16.469 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:16.465+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:16.472 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:16.468+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:16.538 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:16.532+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:16.654 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:16.647+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:16.931 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:16.925+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:16.936 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:16.931+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:16.972 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:16.966+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:17.442 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:17.438+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:17.514 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:17.509+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:17.521 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:17.515+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:17.671 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:17.666+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:17.913 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:17.908+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:17.928 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:17.922+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:18.005 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:17.999+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:18.419 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:18.414+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:18.502 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:18.494+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:18.528 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:18.523+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:18.633 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:18.626+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:18.897 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:18.893+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:18.954 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:18.947+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:19.037 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:19.030+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:19.390 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:19.386+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:19.507 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:19.501+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:19.531 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:19.526+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:19.605 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:19.598+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:19.901 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:19.897+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:19.965 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:19.959+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:20.054 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:20.047+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:20.420 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:20.415+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:20.465 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:20.458+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:20.578 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:20.570+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:20.580 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:20.576+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:20.893 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:20.890+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:20.933 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:20.926+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:21.076 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:21.068+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:21.384 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:21.381+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:21.458 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:21.452+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:21.547 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:21.540+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:21.600 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:21.596+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:21.889 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:21.883+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:21.891 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:21.887+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:22.056 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:22.050+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:22.364 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:22.359+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:22.465 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:22.458+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:22.512 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:22.506+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:22.580 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:22.577+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:22.859 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:22.855+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:22.953 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:22.931+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:23.056 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:23.049+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:23.374 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:23.367+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:23.430 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:23.422+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:23.544 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:23.537+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:23.603 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:23.599+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:23.866 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:23.863+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:23.940 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:23.933+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:24.079 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:24.074+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:24.332 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:24.329+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:24.425 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:24.418+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:24.503 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:24.496+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:24.561 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:24.557+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:24.900 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:24.880+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:24.986 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:24.977+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:25.055 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:25.049+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:25.314 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:25.309+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:25.453 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:25.446+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:25.503 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:25.496+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:25.581 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:25.576+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:25.868 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:25.863+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:26.032 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:26.026+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:26.067 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:26.061+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:26.341 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:26.336+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:26.458 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:26.451+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:26.518 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:26.512+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:26.587 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:26.582+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:26.611 INFO:teuthology.orchestra.run.smithi033.stderr:testBadHeaders (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:22:26.720 INFO:teuthology.orchestra.run.smithi033.stderr:testBlankMetadataName (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:22:26.883 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:26.878+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:27.078 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:27.072+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:27.088 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:27.082+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:27.388 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:27.382+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:27.486 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:27.479+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:27.560 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:27.552+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:27.604 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:27.600+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:27.907 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:27.901+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:28.096 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:28.090+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:28.118 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:28.111+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:28.440 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:28.435+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:28.505 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:28.497+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:28.576 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:28.569+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:28.592 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:28.587+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:28.948 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:28.943+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:29.066 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:29.060+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:29.159 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:29.152+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:29.464 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:29.459+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:29.536 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:29.530+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:29.541 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:29.534+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:29.592 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:29.587+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:29.943 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:29.939+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:30.112 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:30.106+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:30.133 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:22:30.127+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 44sec e299 v299) 2019-12-09T15:22:30.152 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:30.145+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:30.515 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:30.510+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:30.535 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:30.528+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:30.546 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:30.540+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:30.572 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:30.567+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:30.968 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:30.963+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:31.160 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:31.152+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:31.164 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:31.157+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:31.511 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:31.505+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:31.529 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:31.524+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:31.553 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:31.548+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:31.618 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:31.614+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:31.955 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:31.945+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:32.146 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:32.140+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:32.164 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:32.157+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:32.504 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:32.499+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:32.535 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:32.528+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:32.558 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:32.551+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:32.634 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:32.630+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:32.957 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:32.953+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:33.141 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:33.135+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:33.199 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:33.192+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:33.490 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:33.486+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:33.551 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:33.544+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:33.562 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:33.556+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:33.646 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:33.641+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:33.990 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:33.985+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:34.117 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:34.111+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:34.208 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:34.201+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:34.490 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:22:34.486+0000 1694f700 -1 osd.6 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.323678+0000 front 2019-12-09T15:21:12.324879+0000 (oldest deadline 2019-12-09T15:21:55.149369+0000) 2019-12-09T15:22:34.534 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:22:34.527+0000 1694f700 -1 osd.1 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.569468+0000 front 2019-12-09T15:21:10.578097+0000 (oldest deadline 2019-12-09T15:21:55.090601+0000) 2019-12-09T15:22:34.547 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:22:34.540+0000 1694f700 -1 osd.0 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.209367+0000 front 2019-12-09T15:21:12.258763+0000 (oldest deadline 2019-12-09T15:21:54.482283+0000) 2019-12-09T15:22:34.658 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:22:34.653+0000 1694f700 -1 osd.7 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.761962+0000 front 2019-12-09T15:21:10.763668+0000 (oldest deadline 2019-12-09T15:21:54.227876+0000) 2019-12-09T15:22:35.014 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:22:35.009+0000 1694f700 -1 osd.4 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:12.664065+0000 front 2019-12-09T15:21:12.794956+0000 (oldest deadline 2019-12-09T15:21:53.570001+0000) 2019-12-09T15:22:35.098 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:22:35.092+0000 1694f700 -1 osd.2 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:10.923120+0000 front 2019-12-09T15:21:10.921426+0000 (oldest deadline 2019-12-09T15:21:54.840432+0000) 2019-12-09T15:22:35.191 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:22:35.185+0000 1694f700 -1 osd.3 299 heartbeat_check: no reply from 172.21.15.186:6804 osd.5 since back 2019-12-09T15:21:13.003475+0000 front 2019-12-09T15:21:09.987706+0000 (oldest deadline 2019-12-09T15:21:52.921389+0000) 2019-12-09T15:22:35.221 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:22:35.215+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.5 [v2:172.21.15.186:6800/10129,v1:172.21.15.186:6801/10129] for 44sec e299 v299) 2019-12-09T15:23:12.788 INFO:tasks.ceph.osd.5.smithi186.stderr:*** Caught signal (Segmentation fault) ** 2019-12-09T15:23:12.789 INFO:tasks.ceph.osd.5.smithi186.stderr: in thread 124b0700 thread_name:msgr-worker-2 2019-12-09T15:23:12.842 INFO:tasks.ceph.osd.5.smithi186.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T15:23:12.842 INFO:tasks.ceph.osd.5.smithi186.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T15:23:12.842 INFO:tasks.ceph.osd.5.smithi186.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T15:23:12.842 INFO:tasks.ceph.osd.5.smithi186.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T15:23:12.843 INFO:tasks.ceph.osd.5.smithi186.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T15:23:12.843 INFO:tasks.ceph.osd.5.smithi186.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T15:23:12.843 INFO:tasks.ceph.osd.5.smithi186.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T15:23:12.843 INFO:tasks.ceph.osd.5.smithi186.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T15:23:12.843 INFO:tasks.ceph.osd.5.smithi186.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T15:23:12.843 INFO:tasks.ceph.osd.5.smithi186.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T15:23:12.843 INFO:tasks.ceph.osd.5.smithi186.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T15:23:12.844 INFO:tasks.ceph.osd.5.smithi186.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T15:23:12.844 INFO:tasks.ceph.osd.5.smithi186.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T15:23:12.844 INFO:tasks.ceph.osd.5.smithi186.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T15:23:12.864 INFO:tasks.ceph.osd.5.smithi186.stderr:2019-12-09T15:23:12.859+0000 124b0700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T15:23:12.865 INFO:tasks.ceph.osd.5.smithi186.stderr: in thread 124b0700 thread_name:msgr-worker-2 2019-12-09T15:23:12.865 INFO:tasks.ceph.osd.5.smithi186.stderr: 2019-12-09T15:23:12.865 INFO:tasks.ceph.osd.5.smithi186.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T15:23:12.866 INFO:tasks.ceph.osd.5.smithi186.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T15:23:12.866 INFO:tasks.ceph.osd.5.smithi186.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T15:23:12.866 INFO:tasks.ceph.osd.5.smithi186.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T15:23:12.866 INFO:tasks.ceph.osd.5.smithi186.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T15:23:12.866 INFO:tasks.ceph.osd.5.smithi186.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T15:23:12.867 INFO:tasks.ceph.osd.5.smithi186.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T15:23:12.867 INFO:tasks.ceph.osd.5.smithi186.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T15:23:12.867 INFO:tasks.ceph.osd.5.smithi186.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T15:23:12.867 INFO:tasks.ceph.osd.5.smithi186.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T15:23:12.868 INFO:tasks.ceph.osd.5.smithi186.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T15:23:12.868 INFO:tasks.ceph.osd.5.smithi186.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T15:23:12.868 INFO:tasks.ceph.osd.5.smithi186.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T15:23:12.868 INFO:tasks.ceph.osd.5.smithi186.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T15:23:12.869 INFO:tasks.ceph.osd.5.smithi186.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T15:23:12.869 INFO:tasks.ceph.osd.5.smithi186.stderr: 2019-12-09T15:23:13.466 INFO:tasks.ceph.osd.5.smithi186.stderr: -8> 2019-12-09T15:23:12.859+0000 124b0700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T15:23:13.467 INFO:tasks.ceph.osd.5.smithi186.stderr: in thread 124b0700 thread_name:msgr-worker-2 2019-12-09T15:23:13.467 INFO:tasks.ceph.osd.5.smithi186.stderr: 2019-12-09T15:23:13.467 INFO:tasks.ceph.osd.5.smithi186.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T15:23:13.467 INFO:tasks.ceph.osd.5.smithi186.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T15:23:13.468 INFO:tasks.ceph.osd.5.smithi186.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T15:23:13.468 INFO:tasks.ceph.osd.5.smithi186.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T15:23:13.468 INFO:tasks.ceph.osd.5.smithi186.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T15:23:13.468 INFO:tasks.ceph.osd.5.smithi186.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T15:23:13.469 INFO:tasks.ceph.osd.5.smithi186.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T15:23:13.469 INFO:tasks.ceph.osd.5.smithi186.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T15:23:13.469 INFO:tasks.ceph.osd.5.smithi186.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T15:23:13.469 INFO:tasks.ceph.osd.5.smithi186.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T15:23:13.469 INFO:tasks.ceph.osd.5.smithi186.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T15:23:13.469 INFO:tasks.ceph.osd.5.smithi186.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T15:23:13.470 INFO:tasks.ceph.osd.5.smithi186.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T15:23:13.470 INFO:tasks.ceph.osd.5.smithi186.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T15:23:13.471 INFO:tasks.ceph.osd.5.smithi186.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T15:23:13.473 INFO:tasks.ceph.osd.5.smithi186.stderr: 2019-12-09T15:23:13.924 INFO:tasks.ceph.osd.5.smithi186.stderr: -1009> 2019-12-09T15:23:12.859+0000 124b0700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T15:23:13.924 INFO:tasks.ceph.osd.5.smithi186.stderr: in thread 124b0700 thread_name:msgr-worker-2 2019-12-09T15:23:13.924 INFO:tasks.ceph.osd.5.smithi186.stderr: 2019-12-09T15:23:13.924 INFO:tasks.ceph.osd.5.smithi186.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T15:23:13.924 INFO:tasks.ceph.osd.5.smithi186.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T15:23:13.925 INFO:tasks.ceph.osd.5.smithi186.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T15:23:13.925 INFO:tasks.ceph.osd.5.smithi186.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T15:23:13.925 INFO:tasks.ceph.osd.5.smithi186.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T15:23:13.925 INFO:tasks.ceph.osd.5.smithi186.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T15:23:13.925 INFO:tasks.ceph.osd.5.smithi186.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T15:23:13.925 INFO:tasks.ceph.osd.5.smithi186.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T15:23:13.925 INFO:tasks.ceph.osd.5.smithi186.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T15:23:13.926 INFO:tasks.ceph.osd.5.smithi186.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T15:23:13.926 INFO:tasks.ceph.osd.5.smithi186.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T15:23:13.926 INFO:tasks.ceph.osd.5.smithi186.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T15:23:13.926 INFO:tasks.ceph.osd.5.smithi186.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T15:23:13.926 INFO:tasks.ceph.osd.5.smithi186.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T15:23:13.926 INFO:tasks.ceph.osd.5.smithi186.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T15:23:13.927 INFO:tasks.ceph.osd.5.smithi186.stderr: 2019-12-09T15:23:17.102 INFO:tasks.ceph.osd.5.smithi186.stderr:daemon-helper: command crashed with signal 11 2019-12-09T15:23:21.714 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2019-12-09T15:23:28.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~6s 2019-12-09T15:23:32.906 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:32.894+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:33.758 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:33.752+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:33.879 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:33.873+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:34.321 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:34.315+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:34.329 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~13s 2019-12-09T15:23:34.720 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:34.713+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:34.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:34.750+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:34.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:34.846+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:35.291 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:35.285+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:35.702 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:35.696+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:35.776 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:35.769+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:35.809 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:35.802+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:36.058 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:36.051+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:36.304 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:36.296+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:36.679 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:36.674+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:36.778 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:36.772+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:36.806 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:36.800+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:36.932 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:36.926+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:37.083 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:37.077+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:37.345 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:37.340+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:37.722 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:37.716+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:37.799 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:37.794+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:37.813 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:37.807+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:37.964 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:37.959+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:38.062 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:38.055+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:38.302 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:38.296+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:38.720 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:38.714+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:38.829 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:38.823+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:38.853 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:38.848+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:39.020 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:39.015+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:39.097 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:39.091+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:39.271 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:39.266+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:39.729 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:39.723+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:39.839 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:39.833+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:39.840 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:39.834+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:39.984 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:39.978+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:40.082 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:40.076+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:40.252 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:40.247+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:40.640 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~19s 2019-12-09T15:23:40.733 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:40.728+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:40.882 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:40.877+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:40.904 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:40.883+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:41.013 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:41.008+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:41.092 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:41.086+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:41.248 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:41.243+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:41.702 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:41.696+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:41.855 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:41.848+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:41.926 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:41.920+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:42.034 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:42.029+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:42.139 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:42.134+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:42.273 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:42.268+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:42.734 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:42.728+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:42.832 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:42.825+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:42.923 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:42.918+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:43.003 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:42.993+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:43.142 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:43.136+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:43.268 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:43.262+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:43.760 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:43.755+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:43.871 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:43.865+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:43.911 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:43.904+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:43.956 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:43.951+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:44.106 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:44.100+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:44.264 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:44.259+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:44.752 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:44.745+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:44.888 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:44.882+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:44.965 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:44.959+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:44.999 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:44.994+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:45.076 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:45.072+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:45.222 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:45.216+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:45.780 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:45.774+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:45.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:45.863+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:45.992 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:45.987+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:46.010 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:46.004+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:46.123 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:46.117+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:46.199 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:46.193+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:46.824 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:46.817+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:46.885 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:46.878+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:46.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~25s 2019-12-09T15:23:47.010 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:47.004+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:47.047 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:47.041+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:47.133 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:47.127+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:47.251 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:47.245+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:47.828 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:47.821+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:47.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:47.858+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:48.002 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:47.996+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:48.060 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:48.054+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:48.173 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:48.167+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:48.272 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:48.266+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:48.814 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:48.808+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:48.847 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:48.841+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:48.961 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:48.955+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:49.088 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:49.083+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:49.177 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:49.172+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:49.279 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:49.275+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:49.809 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:49.804+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:49.825 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:49.819+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:49.971 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:49.966+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:50.142 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:50.137+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:50.163 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:50.158+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:50.276 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:50.271+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:50.798 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:50.793+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:50.832 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:50.827+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:51.001 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:50.996+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:51.114 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:51.108+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:51.208 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:51.203+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:51.297 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:51.292+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:51.815 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:51.807+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:51.831 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:51.826+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:51.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:51.977+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:52.158 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:52.152+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:52.216 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:52.212+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:52.333 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:52.317+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:52.783 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:52.777+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:52.870 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:52.864+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:52.971 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:52.966+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:53.176 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:53.171+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:53.205 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:53.199+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:53.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~32s 2019-12-09T15:23:53.294 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:53.288+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:53.811 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:53.805+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:53.828 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:53.823+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:53.929 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:53.924+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:54.166 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:54.160+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:54.223 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:54.217+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:54.317 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:54.312+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:54.790 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:54.784+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:54.869 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:54.863+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:54.909 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:54.904+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:55.212 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:55.207+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:55.248 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:55.242+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:55.284 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:55.279+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:55.775 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:55.769+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:55.852 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:55.847+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:55.907 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:55.902+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:56.224 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:56.218+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:56.283 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:56.277+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:56.293 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:56.287+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:56.737 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:56.732+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:56.860 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:56.855+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:56.894 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:56.889+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:57.246 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:57.240+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:57.289 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:57.284+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:57.292 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:57.287+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:57.763 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:57.758+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:57.901 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:57.896+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:57.945 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:57.939+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:58.234 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:58.228+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:58.276 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:58.271+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:58.298 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:58.292+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:58.735 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:58.729+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:58.893 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:58.887+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:58.972 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:58.966+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:23:59.205 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:23:59.199+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:23:59.253 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:23:59.247+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:23:59.285 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:23:59.280+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:23:59.566 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~38s 2019-12-09T15:23:59.787 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:23:59.781+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:23:59.883 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:23:59.877+0000 1694f700 -1 osd.3 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.397009+0000 front 2019-12-09T15:22:50.738315+0000 (oldest deadline 2019-12-09T15:23:34.070833+0000) 2019-12-09T15:23:59.938 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:23:59.933+0000 1694f700 -1 osd.0 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.201030+0000 front 2019-12-09T15:22:50.221120+0000 (oldest deadline 2019-12-09T15:23:32.464023+0000) 2019-12-09T15:24:00.225 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:24:00.221+0000 1694f700 -1 osd.4 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:50.223220+0000 front 2019-12-09T15:22:50.255866+0000 (oldest deadline 2019-12-09T15:23:35.495090+0000) 2019-12-09T15:24:00.306 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:24:00.295+0000 1694f700 -1 osd.6 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.862666+0000 front 2019-12-09T15:22:51.806300+0000 (oldest deadline 2019-12-09T15:23:33.786569+0000) 2019-12-09T15:24:00.308 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:24:00.302+0000 1694f700 -1 osd.7 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.379285+0000 front 2019-12-09T15:22:51.855753+0000 (oldest deadline 2019-12-09T15:23:36.031502+0000) 2019-12-09T15:24:00.801 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:24:00.796+0000 1694f700 -1 osd.1 302 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:22:51.298450+0000 front 2019-12-09T15:22:51.859620+0000 (oldest deadline 2019-12-09T15:23:33.570289+0000) 2019-12-09T15:24:05.877 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~44s 2019-12-09T15:24:12.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~50s 2019-12-09T15:24:18.497 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~57s 2019-12-09T15:24:24.804 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~63s 2019-12-09T15:24:31.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~69s 2019-12-09T15:24:37.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~76s 2019-12-09T15:24:43.736 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~82s 2019-12-09T15:24:50.046 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~88s 2019-12-09T15:24:56.022 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:56:17.481 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.073 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:56:17.533 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.097 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:56:17.556 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.124 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:56:17.584 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.161 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:56:17.642 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.190 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:56:17.670 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.346 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:56:17.805 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~95s 2019-12-09T15:24:56.374 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:56:17.833 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:56.453 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:56:17.934 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:25:02.664 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~101s 2019-12-09T15:25:08.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~107s 2019-12-09T15:25:15.284 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~114s 2019-12-09T15:25:21.588 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~120s 2019-12-09T15:25:27.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~126s 2019-12-09T15:25:34.200 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~132s 2019-12-09T15:25:40.506 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~139s 2019-12-09T15:25:44.971 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:44.965+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:45.754 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:45.748+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:45.971 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:45.965+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:46.073 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:46.068+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:46.622 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:46.617+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:46.685 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:46.679+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:46.798 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:46.793+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:46.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~145s 2019-12-09T15:25:46.980 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:46.974+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:47.033 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:47.027+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:47.607 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:47.601+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:47.648 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:47.642+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:47.799 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:47.793+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:47.996 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:47.990+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:48.008 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:48.002+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:48.609 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:48.603+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:48.680 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:48.675+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:48.854 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:48.848+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:48.992 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:48.987+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:49.052 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:49.047+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:49.581 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:49.575+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:49.707 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:49.701+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:49.850 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:49.844+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:49.986 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:49.981+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:50.005 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:49.998+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:50.035 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:50.029+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:50.632 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:50.625+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:50.716 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:50.710+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:50.814 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:50.809+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:50.996 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:50.990+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:51.017 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:51.010+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:51.044 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:51.039+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:51.611 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:51.604+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:51.756 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:51.750+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:51.808 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:51.803+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:51.988 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:51.982+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:52.012 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:52.006+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:52.047 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:52.041+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:52.606 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:52.602+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:52.799 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:52.794+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:52.854 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:52.849+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:53.007 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:53.002+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:53.019 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:53.013+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:53.066 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:53.058+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:53.126 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~151s 2019-12-09T15:25:53.642 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:53.637+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:53.784 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:53.778+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:53.904 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:53.899+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:53.969 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:53.963+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:54.031 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:54.024+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:54.086 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:54.081+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:54.664 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:54.657+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:54.837 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:54.831+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:54.918 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:54.904+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:54.976 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:54.971+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:55.064 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:55.059+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:55.093 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:55.088+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:55.698 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:55.692+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:55.853 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:55.848+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:55.940 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:55.935+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:55.966 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:55.961+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:56.033 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:56.028+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:56.063 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:56.056+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:56.710 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:56.704+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:56.822 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:56.817+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:56.989 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:56.980+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:56.999 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:56.994+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:57.028 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:57.021+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:57.053 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:57.048+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:57.731 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:57.726+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:57.850 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:57.846+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:57.960 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:57.956+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:58.006 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:58.000+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:58.037 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:58.031+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:58.041 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:58.037+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:58.719 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:58.713+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:58.859 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:58.852+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:59.011 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:59.007+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:25:59.036 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:25:59.030+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:25:59.053 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:25:59.047+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:25:59.060 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:25:59.056+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:25:59.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~158s 2019-12-09T15:25:59.769 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:25:59.763+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:25:59.827 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:25:59.823+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:25:59.991 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:25:59.987+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:00.027 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:00.021+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:00.032 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:00.025+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:00.035 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:00.031+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:00.761 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:00.756+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:00.856 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:00.852+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:00.993 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:00.988+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:00.993 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:00.989+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:01.007 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:01.001+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:01.066 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:01.061+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:01.788 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:01.782+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:01.874 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:01.869+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:02.034 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:02.030+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:02.042 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:02.038+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:02.061 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:02.055+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:02.069 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:02.064+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:02.786 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:02.780+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:02.861 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:02.856+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:03.035 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:03.030+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:03.038 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:03.031+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:03.041 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:03.035+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:03.050 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:03.044+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:03.773 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:03.767+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:03.891 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:03.885+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:04.035 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:04.029+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:04.048 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:04.041+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:04.052 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:04.046+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:04.075 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:04.069+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:04.774 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:04.769+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:04.873 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:04.861+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:05.049 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:05.043+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:05.076 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:05.071+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:05.083 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:05.077+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:05.089 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:05.084+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:05.741 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~164s 2019-12-09T15:26:05.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:05.749+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:05.827 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:05.822+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:06.020 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:06.015+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:06.042 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:06.036+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:06.060 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:06.054+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:06.110 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:06.104+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:06.297 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:57:27.756 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.311 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:57:27.770 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.325 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:57:27.785 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.330 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:57:27.790 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.336 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:57:27.818 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.351 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:57:27.833 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.402 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:05:57:27.862 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.419 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:05:57:27.878 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.525 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:05:57:28.006 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.784 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:06.778+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:06.815 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:06.798+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:07.039 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:07.033+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:07.042 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:07.037+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:07.055 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:07.050+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:07.071 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:07.066+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:07.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:07.754+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:07.802 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:07.797+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:08.029 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:08.024+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:08.029 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:08.023+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:08.063 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:08.057+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:08.101 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:08.095+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:08.786 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:08.780+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:08.801 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:08.796+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:08.993 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:08.988+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:09.072 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:09.066+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:09.079 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:09.074+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:09.081 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:09.075+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:09.753 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:09.748+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:09.810 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:09.805+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:10.002 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:09.997+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:10.038 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:10.031+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:10.080 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:10.074+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:10.088 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:10.081+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:10.726 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:10.720+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:10.809 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:10.803+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:11.016 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:11.002+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:11.031 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:11.025+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:11.050 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:11.044+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:11.093 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:11.087+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:11.727 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:11.720+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:11.839 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:11.833+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:11.996 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:11.989+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:12.013 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:12.006+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:12.049 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~170s 2019-12-09T15:26:12.051 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:12.044+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:12.112 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:12.106+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:12.688 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:12.682+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:12.834 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:12.827+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:12.951 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:12.944+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:13.019 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:13.013+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:13.057 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:13.051+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:13.131 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:13.124+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:13.640 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:13.635+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:13.845 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:13.840+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:13.921 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:13.914+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:14.046 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:14.040+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:14.073 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:14.067+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:14.135 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:14.130+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:14.603 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:14.597+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:14.833 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:14.818+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:14.964 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:14.959+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:15.002 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:14.996+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:15.115 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:15.110+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:15.133 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:15.128+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:15.593 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:26:15.586+0000 1694f700 -1 osd.3 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:00.490559+0000 front 2019-12-09T15:25:00.488350+0000 (oldest deadline 2019-12-09T15:25:45.674083+0000) 2019-12-09T15:26:15.865 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:26:15.858+0000 1694f700 -1 osd.4 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.003034+0000 front 2019-12-09T15:25:03.127034+0000 (oldest deadline 2019-12-09T15:25:45.877500+0000) 2019-12-09T15:26:15.940 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:26:15.934+0000 1694f700 -1 osd.0 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:03.466943+0000 front 2019-12-09T15:25:03.873531+0000 (oldest deadline 2019-12-09T15:25:44.841054+0000) 2019-12-09T15:26:16.012 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:26:16.006+0000 1694f700 -1 osd.1 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.877329+0000 front 2019-12-09T15:25:02.720022+0000 (oldest deadline 2019-12-09T15:25:49.001825+0000) 2019-12-09T15:26:16.074 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:26:16.069+0000 1694f700 -1 osd.6 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:01.631551+0000 front 2019-12-09T15:25:01.619322+0000 (oldest deadline 2019-12-09T15:25:45.102131+0000) 2019-12-09T15:26:16.099 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:26:16.094+0000 1694f700 -1 osd.7 307 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:25:02.239574+0000 front 2019-12-09T15:25:02.246442+0000 (oldest deadline 2019-12-09T15:25:45.117207+0000) 2019-12-09T15:26:18.358 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~177s 2019-12-09T15:26:24.664 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~183s 2019-12-09T15:26:30.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~189s 2019-12-09T15:26:37.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~196s 2019-12-09T15:26:43.601 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~202s 2019-12-09T15:26:49.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~208s 2019-12-09T15:26:56.226 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~215s 2019-12-09T15:27:02.533 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~221s 2019-12-09T15:27:08.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~227s 2019-12-09T15:27:15.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~233s 2019-12-09T15:27:21.468 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~240s 2019-12-09T15:27:27.784 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~246s 2019-12-09T15:27:34.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~252s 2019-12-09T15:27:40.417 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~259s 2019-12-09T15:27:46.730 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~265s 2019-12-09T15:27:53.038 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~271s 2019-12-09T15:27:59.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~278s 2019-12-09T15:28:00.544 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:00.538+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:00.627 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:00.621+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:00.751 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:00.745+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:01.597 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:01.592+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:01.639 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:01.633+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:01.746 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:01.740+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:02.568 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:02.563+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:02.615 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:02.608+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:02.785 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:02.779+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:03.249 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:03.241+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:03.261 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:03.244+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:03.561 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:03.547+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:03.585 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:03.577+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:03.787 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:03.782+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:04.214 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:04.209+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:04.289 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:04.284+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:04.570 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:04.563+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:04.591 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:04.584+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:04.723 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:04.718+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:04.777 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:04.772+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:05.241 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:05.235+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:05.306 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:05.301+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:05.548 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:05.540+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:05.621 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:05.615+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:05.659 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~284s 2019-12-09T15:28:05.703 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:05.697+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:05.746 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:05.741+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:06.216 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:06.210+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:06.294 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:06.288+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:06.505 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:06.498+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:06.612 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:06.607+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:06.712 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:06.706+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:06.795 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:06.790+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:07.182 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:07.177+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:07.286 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:07.279+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:07.486 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:07.477+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:07.653 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:07.648+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:07.668 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:07.662+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:07.776 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:07.771+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:08.158 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:08.153+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:08.303 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:08.297+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:08.467 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:08.460+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:08.664 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:08.658+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:08.691 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:08.686+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:08.798 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:08.794+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:09.174 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:09.169+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:09.309 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:09.302+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:09.484 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:09.479+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:09.634 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:09.629+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:09.681 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:09.675+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:09.785 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:09.780+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:10.201 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:10.194+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:10.277 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:10.270+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:10.533 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:10.526+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:10.611 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:10.606+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:10.729 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:10.723+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:10.791 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:10.786+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:11.199 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:11.194+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:11.318 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:11.312+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:11.521 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:11.515+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:11.585 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:11.580+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:11.700 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:11.694+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:11.841 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:11.835+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:11.978 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~290s 2019-12-09T15:28:12.212 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:12.205+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:12.339 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:12.333+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:12.521 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:12.515+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:12.635 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:12.630+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:12.699 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:12.693+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:12.854 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:12.848+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:13.167 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:13.161+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:13.334 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:13.328+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:13.566 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:13.560+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:13.625 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:13.620+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:13.712 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:13.706+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:13.844 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:13.839+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:14.123 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:14.117+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:14.307 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:14.302+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:14.580 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:14.574+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:14.638 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:14.632+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:14.691 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:14.685+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:14.847 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:14.841+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:15.109 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:15.104+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:15.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:15.356+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:15.606 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:15.599+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:15.666 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:15.660+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:15.701 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:15.695+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:15.863 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:15.858+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:16.082 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:16.077+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:16.336 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:16.329+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:16.623 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:16.617+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:16.638 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:16.631+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:16.687 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:16.681+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:16.897 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:16.891+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:17.093 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:17.086+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:17.300 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:17.294+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:17.583 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:17.577+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:17.633 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:17.627+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:17.736 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:17.731+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:17.930 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:17.925+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:18.047 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:18.040+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:18.279 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:18.274+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:18.289 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~297s 2019-12-09T15:28:18.607 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:18.601+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:18.622 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:18.616+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:18.742 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:18.737+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:18.975 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:18.968+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:19.026 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:19.019+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:19.312 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:19.306+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:19.654 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:19.648+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:19.655 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:19.649+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:19.747 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:19.742+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:19.985 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:19.978+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:19.995 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:19.989+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:20.310 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:20.305+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:20.633 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:20.628+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:20.635 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:20.628+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:20.769 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:20.764+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:20.985 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:20.979+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:21.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:21.006+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:21.331 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:21.326+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:21.600 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:21.594+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:21.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:21.659+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:21.806 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:21.799+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:21.989 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:21.984+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:22.017 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:22.010+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:22.352 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:22.337+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:22.599 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:22.591+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:22.701 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:22.691+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:22.778 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:22.772+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:23.000 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:22.992+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:23.041 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:23.035+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:23.329 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:23.323+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:23.563 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:23.557+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:23.690 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:23.684+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:23.761 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:23.756+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:24.026 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:24.019+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:24.084 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:24.079+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:24.337 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:24.331+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:24.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~303s 2019-12-09T15:28:24.596 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2019-12-09T15:28:24.596 ERROR:tasks.daemonwatchdog.daemon_watchdog:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 38, in _run self.watch() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 114, in watch self.bark() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 54, in bark for mount in self.ctx.mounts.values(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/config.py", line 253, in __getattr__ raise AttributeError(name) AttributeError: mounts 2019-12-09T15:28:24.615 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:24.609+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:24.652 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:24.646+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:24.746 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:24.735+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:25.016 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:25.011+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:25.112 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:25.105+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:25.334 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:25.327+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:25.632 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:25.626+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:25.645 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:25.639+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:25.765 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:25.760+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:25.981 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:25.972+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:26.164 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:26.159+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:26.380 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:26.373+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:26.674 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:26.667+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:26.685 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:26.679+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:26.776 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:26.770+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:27.022 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:27.014+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:27.128 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:27.122+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:27.416 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:27.409+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:27.678 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:27.672+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:27.698 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:27.691+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:27.749 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:27.743+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:28.037 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:28.030+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:28.138 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:28.132+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:28.445 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:28.439+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:28.670 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:28.664+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:28.673 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:28.667+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:28.781 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:28.775+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:29.004 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:28.998+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:29.125 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:29.120+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:29.430 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:29.423+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:29.644 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:29.636+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:29.716 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:29.710+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:29.749 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:29.744+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:29.987 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:29.980+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:30.092 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:30.086+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:30.419 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:30.412+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:30.610 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:30.602+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:30.770 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:30.764+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:30.771 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:30.766+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:31.032 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:31.025+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:31.063 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:31.058+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:31.441 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:31.433+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:31.616 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:31.610+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:31.778 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:28:31.770+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e313 v313) 2019-12-09T15:28:31.785 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:31.779+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:31.820 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:31.814+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:32.002 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:31.994+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:32.074 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:32.068+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:32.417 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:32.410+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:32.573 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:32.564+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:32.765 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:32.759+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:32.867 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:32.861+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:33.018 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:33.011+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:33.040 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:33.035+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:33.398 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:33.391+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:33.543 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:33.536+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:33.741 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:33.736+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:33.826 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:33.818+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:34.028 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:34.022+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:34.071 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:34.064+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:34.432 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:34.426+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:34.532 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:34.525+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:34.720 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:34.714+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:34.790 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:34.785+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:35.004 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:34.998+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:35.065 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:35.059+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:35.411 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:35.403+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:35.564 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:35.558+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:35.742 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:35.736+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:35.838 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:35.832+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:36.024 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:36.017+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:36.044 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:36.039+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:28:36.440 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:28:36.434+0000 1694f700 -1 osd.3 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.909367+0000 front 2019-12-09T15:27:17.969832+0000 (oldest deadline 2019-12-09T15:28:02.555892+0000) 2019-12-09T15:28:36.534 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:28:36.527+0000 1694f700 -1 osd.1 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.953848+0000 front 2019-12-09T15:27:17.969438+0000 (oldest deadline 2019-12-09T15:28:00.205188+0000) 2019-12-09T15:28:36.712 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:28:36.705+0000 1694f700 -1 osd.4 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.233178+0000 front 2019-12-09T15:27:14.235382+0000 (oldest deadline 2019-12-09T15:27:59.827609+0000) 2019-12-09T15:28:36.826 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:28:36.815+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e313 v313) 2019-12-09T15:28:36.872 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:28:36.867+0000 1694f700 -1 osd.6 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:18.638953+0000 front 2019-12-09T15:27:18.650506+0000 (oldest deadline 2019-12-09T15:28:04.518019+0000) 2019-12-09T15:28:37.008 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:28:37.001+0000 1694f700 -1 osd.0 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:17.264937+0000 front 2019-12-09T15:27:17.203473+0000 (oldest deadline 2019-12-09T15:28:03.033114+0000) 2019-12-09T15:28:37.037 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:28:37.033+0000 1694f700 -1 osd.7 313 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:27:14.939056+0000 front 2019-12-09T15:27:15.016767+0000 (oldest deadline 2019-12-09T15:28:00.205225+0000) 2019-12-09T15:29:12.139 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:00:33.597 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.143 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:00:33.603 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.158 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:00:33.618 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.161 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:00:33.620 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.178 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:00:33.659 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.182 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:00:33.663 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.325 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:00:33.784 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.389 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:00:33.848 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:12.407 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:00:33.888 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:29:54.336 INFO:teuthology.orchestra.run.smithi033.stderr:testChunkedPut (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:00.878 INFO:teuthology.orchestra.run.smithi033.stderr:testContentTypeGuessing (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:11.381 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:11.374+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:11.855 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:11.849+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:12.089 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:12.079+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:12.410 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:12.389+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:12.429 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:01:33.889 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.443 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:01:33.902 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.446 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:01:33.928 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.446 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:01:33.906 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.448 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:01:33.908 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.467 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:01:33.949 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.541 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:01:34.001 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.553 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:01:34.013 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.583 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:01:34.064 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:30:12.835 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:12.829+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:13.119 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:13.113+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:13.121 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:13.115+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:13.456 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:13.451+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:13.824 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:13.818+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:14.080 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:14.075+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:14.139 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:14.135+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:14.145 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:14.138+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:14.448 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:14.443+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:14.783 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:14.778+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:15.057 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:15.051+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:15.140 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:15.135+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:15.172 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:15.163+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:15.406 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:15.401+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:15.756 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:15.750+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:15.987 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:15.980+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:16.050 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:16.044+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:16.176 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:16.170+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:16.195 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:16.189+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:16.387 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:16.381+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:16.775 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:16.770+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:17.027 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:17.021+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:17.038 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:17.033+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:17.141 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:17.136+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:17.180 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:17.174+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:17.286 INFO:teuthology.orchestra.run.smithi033.stderr:testCopy (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:17.374 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:17.369+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:17.792 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:17.786+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:18.036 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:18.029+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:18.045 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:18.037+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:18.158 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:18.153+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:18.171 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:18.165+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:18.362 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:18.355+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:18.774 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:18.768+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:19.021 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:19.016+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:19.022 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:19.015+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:19.125 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:19.120+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:19.220 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:19.214+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:19.344 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:19.339+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:19.759 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:19.754+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:20.042 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:20.036+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:20.061 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:20.056+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:20.162 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:20.155+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:20.232 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:20.222+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:20.300 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:20.294+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:20.798 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:20.794+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:21.057 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:21.052+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:21.059 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:21.053+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:21.176 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:21.171+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:21.241 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:21.234+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:21.315 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:21.310+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:21.845 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:21.840+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:22.060 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:22.054+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:22.093 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:22.088+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:22.189 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:22.184+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:22.261 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:22.255+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:22.282 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:22.276+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:22.841 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:22.836+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:23.088 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:23.083+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:23.094 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:23.089+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:23.190 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:23.185+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:23.271 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:23.266+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:23.300 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:23.294+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:23.843 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:23.838+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:24.066 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:24.060+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:24.083 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:24.077+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:24.198 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:24.192+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:24.280 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:24.274+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:24.288 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:24.282+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:24.808 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:24.801+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:25.068 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:25.062+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:25.068 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:25.062+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:25.167 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:25.162+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:25.314 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:25.306+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:25.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:25.319+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:25.781 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:25.776+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:26.060 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:26.054+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:26.072 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:26.066+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:26.126 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:26.121+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:26.152 INFO:teuthology.orchestra.run.smithi033.stderr:testCopy404s (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:26.306 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:26.301+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:26.308 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:26.303+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:26.742 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:26.736+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:27.048 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:27.043+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:27.097 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:27.091+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:27.097 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:27.092+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:27.343 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:27.335+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:27.350 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:27.344+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyDestinationSlashProblems (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:27.794 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:27.788+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:28.077 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:28.072+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:28.099 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:28.090+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:28.115 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:28.109+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:28.371 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:28.365+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:28.411 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:28.405+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:28.845 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:28.838+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:29.076 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:29.071+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:29.110 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:29.102+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:29.118 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:29.114+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:29.372 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:29.358+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:29.424 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:29.420+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:29.806 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:29.800+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:30.087 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:30.083+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:30.148 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:30.142+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:30.169 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:30.164+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:30.381 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:30.374+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:30.430 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:30.426+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:30.792 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:30.785+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:31.090 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:31.086+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:31.151 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:31.143+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:31.152 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:31.148+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:31.388 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:31.381+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:31.430 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:31.426+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:31.824 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:31.816+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:32.067 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:32.062+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:32.139 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:32.131+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:32.169 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:32.164+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:32.346 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:32.339+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:32.448 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:32.442+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:32.827 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:32.821+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:33.076 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:33.072+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:33.119 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:33.113+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:33.201 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:33.196+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:33.361 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:33.354+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:33.402 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:33.397+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:33.823 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:33.816+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:34.043 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:34.038+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:34.109 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:34.103+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:34.173 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:34.167+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:34.349 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:34.342+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:34.454 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:34.449+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:34.863 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:34.857+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:35.038 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:35.034+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:35.092 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:35.086+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:35.149 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:35.144+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:35.395 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:35.388+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:35.432 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:35.427+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:35.881 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:35.872+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:36.021 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:36.015+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:36.049 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:36.042+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:36.161 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:36.156+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:36.379 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:36.373+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:36.421 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:36.415+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:36.877 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:36.871+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:37.041 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:37.035+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:37.061 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:37.054+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:37.201 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:37.195+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:37.336 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:37.330+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:37.401 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:37.395+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:37.836 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:37.830+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:38.079 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:38.074+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:38.088 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:38.082+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:38.225 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:38.219+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:38.371 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:38.364+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:38.395 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:38.389+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:38.879 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:38.873+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:39.074 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:39.069+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:39.117 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:39.110+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:39.183 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:39.178+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:39.370 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:39.364+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:39.424 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:39.418+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:39.870 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:39.862+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:40.111 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:40.106+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:40.177 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:40.166+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:40.185 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:40.179+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:40.361 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:40.354+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:40.416 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:40.407+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:40.919 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:40.912+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:41.158 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:41.152+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:41.168 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:41.163+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:41.211 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:41.203+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:41.384 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:41.378+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:41.405 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:41.398+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:41.881 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:41.875+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:42.125 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:42.119+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:42.182 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:42.176+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:42.233 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:42.226+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:42.402 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:42.395+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:42.436 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:42.431+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:42.931 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:42.926+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:43.157 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:43.152+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:43.159 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:43.154+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:43.196 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:43.188+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:43.375 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:43.367+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:43.433 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:43.428+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:43.941 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:43.934+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:44.151 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:44.146+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:44.152 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:44.147+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:44.244 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:44.232+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:44.370 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:44.365+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:44.401 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:44.396+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:44.471 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyFromHeader (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:44.921 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:44.914+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:45.106 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:45.101+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:45.149 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:45.144+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:45.266 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:45.261+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:45.346 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:45.336+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:45.426 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:45.417+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:45.929 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:45.923+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:46.096 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:46.092+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:46.132 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:46.125+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:46.277 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:46.272+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:46.333 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:46.328+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:46.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:46.439+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:46.961 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:30:46.955+0000 1694f700 -1 osd.3 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:29.396344+0000 front 2019-12-09T15:29:29.729263+0000 (oldest deadline 2019-12-09T15:30:10.963146+0000) 2019-12-09T15:30:47.050 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyFromHeader404s (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:47.103 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:30:47.098+0000 1694f700 -1 osd.7 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.471547+0000 front 2019-12-09T15:29:27.430989+0000 (oldest deadline 2019-12-09T15:30:11.455413+0000) 2019-12-09T15:30:47.136 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:30:47.131+0000 1694f700 -1 osd.6 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.279956+0000 front 2019-12-09T15:29:30.288790+0000 (oldest deadline 2019-12-09T15:30:13.753062+0000) 2019-12-09T15:30:47.286 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:30:47.280+0000 1694f700 -1 osd.1 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:30.141408+0000 front 2019-12-09T15:29:30.009833+0000 (oldest deadline 2019-12-09T15:30:15.222996+0000) 2019-12-09T15:30:47.375 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:30:47.369+0000 1694f700 -1 osd.0 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:27.592990+0000 front 2019-12-09T15:29:27.604530+0000 (oldest deadline 2019-12-09T15:30:12.882085+0000) 2019-12-09T15:30:47.385 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:30:47.377+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 43sec e318 v318) 2019-12-09T15:30:47.484 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:30:47.432+0000 1694f700 -1 osd.4 318 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:29:25.561983+0000 front 2019-12-09T15:29:25.455517+0000 (oldest deadline 2019-12-09T15:30:11.277335+0000) 2019-12-09T15:30:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:testCopyNoDestinationHeader (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:52.250 INFO:teuthology.orchestra.run.smithi033.stderr:testDelete (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:52.835 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteOfFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:testDeleteThen404s (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:58.371 INFO:teuthology.orchestra.run.smithi033.stderr:testEtagResponse (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:30:58.505 INFO:teuthology.orchestra.run.smithi033.stderr:testEtagWayoff (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:31:18.153 INFO:teuthology.orchestra.run.smithi033.stderr:testFileCreate (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:31:19.995 INFO:teuthology.orchestra.run.smithi033.stderr:testGetContentType (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:31:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:testGetOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:31:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:testHead (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:31:22.804 INFO:teuthology.orchestra.run.smithi033.stderr:testHeadOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:31:54.687 INFO:teuthology.orchestra.run.smithi033.stderr:testMetadataOnPost (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:32:11.938 INFO:teuthology.orchestra.run.smithi033.stderr:testMetadataOnPut (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:32:18.648 INFO:teuthology.orchestra.run.smithi033.stderr:testNameLimit (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:32:18.676 INFO:teuthology.orchestra.run.smithi033.stderr:testNoContentLengthForPut (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:32:19.630 INFO:teuthology.orchestra.run.smithi033.stderr:testPostOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:32:25.645 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:25.640+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:25.860 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:25.853+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:26.019 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:26.015+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:26.674 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:26.668+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:26.870 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:26.864+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:27.046 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:27.039+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:27.646 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:27.641+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:27.874 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:27.868+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:27.963 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:27.957+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:28.043 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:28.037+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:28.679 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:28.673+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:28.879 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:28.873+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:28.996 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:28.989+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:29.013 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:29.005+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:29.701 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:29.695+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:29.857 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:29.850+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:30.001 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:29.993+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:30.013 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:30.005+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:30.419 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:30.414+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:30.734 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:30.728+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:30.892 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:30.885+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:30.982 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:30.976+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:30.995 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:30.986+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:31.443 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:31.437+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:31.779 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:31.774+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:31.883 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:31.877+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:31.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:31.975+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:32.001 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:31.995+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:32.440 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:32.433+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:32.760 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:32.755+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:32.889 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:32.882+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:32.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:32.975+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:33.017 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:33.010+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:33.410 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:33.404+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:33.758 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:33.750+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:33.924 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:33.917+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:33.945 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:33.939+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:33.996 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:33.988+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:34.405 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:34.400+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:34.740 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:34.734+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:34.915 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:34.908+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:34.974 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:34.968+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:34.990 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:34.984+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:35.448 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:35.443+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:35.742 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:35.736+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:35.929 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:35.922+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:35.985 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:35.979+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:36.008 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:35.999+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:36.416 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:36.410+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:36.764 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:36.759+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:36.894 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:36.888+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:37.011 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:37.000+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:37.013 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:37.006+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:37.388 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:37.383+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:37.781 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:37.775+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:37.883 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:37.876+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:38.042 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:38.033+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:38.063 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:38.053+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:38.375 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:38.369+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:38.783 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:38.773+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:38.925 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:38.918+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:39.051 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:39.044+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:39.083 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:39.076+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:39.392 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:39.387+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:39.790 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:39.785+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:39.941 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:39.934+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:40.098 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:40.090+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:40.098 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:40.091+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:40.399 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:40.393+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:40.775 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:40.769+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:40.990 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:40.982+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:41.084 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:41.078+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:41.087 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:41.080+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:41.446 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:41.441+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:41.826 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:41.820+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:41.972 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:41.966+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:42.055 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:42.048+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:42.134 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:42.128+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:42.496 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:42.491+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:42.801 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:42.794+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:42.961 INFO:teuthology.orchestra.run.smithi033.stderr:testRangedGets (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:32:42.996 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:42.988+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:43.098 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:43.090+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:43.107 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:43.100+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:43.543 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:43.538+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:43.777 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:43.772+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:44.006 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:43.998+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:44.058 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:44.052+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:44.064 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:44.054+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:44.536 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:44.531+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:44.766 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:44.761+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:45.001 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:44.994+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:45.049 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:45.043+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:45.055 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:45.049+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:45.560 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:45.554+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:45.825 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:45.820+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:46.012 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:46.004+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:46.050 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:46.044+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:46.102 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:46.096+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:46.574 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:46.554+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:46.793 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:46.788+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:46.965 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:46.959+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:47.058 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:47.052+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:47.156 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:47.150+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:47.518 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:47.512+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:47.842 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:47.833+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:48.007 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:48.002+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:48.015 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:48.010+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:48.134 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:48.129+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:48.505 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:48.499+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:48.867 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:48.862+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:49.029 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:49.023+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:49.052 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:49.044+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:49.133 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:49.126+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:49.525 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:49.520+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:49.863 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:49.858+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:50.000 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:49.994+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:50.024 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:50.018+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:50.100 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:50.095+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:50.529 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:50.523+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:50.854 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:50.847+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:50.988 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:50.982+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:51.067 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:51.061+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:51.074 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:51.068+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:51.511 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:51.506+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:51.855 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:51.850+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:52.005 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:52.000+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:52.082 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:52.077+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:52.118 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:52.112+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:52.534 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:52.528+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:52.877 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:52.871+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:52.963 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:52.957+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:53.041 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:53.035+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:53.146 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:53.140+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:53.562 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:53.556+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:53.903 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:53.897+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:54.000 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:53.995+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:54.008 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:54.002+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:54.112 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:54.107+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:54.551 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:54.546+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:54.881 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:32:54.874+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:32:54.946 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:54.940+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:54.958 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:54.952+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:54.995 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:54.989+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:55.147 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:55.141+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:55.550 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:55.540+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:55.883 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:32:55.877+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:32:55.922 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:55.914+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:55.989 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:55.981+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:56.011 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:56.004+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:56.186 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:56.181+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:56.515 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:56.509+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:56.912 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:32:56.906+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:32:56.946 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:56.940+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:56.968 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:56.961+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:57.011 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:57.005+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:57.149 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:57.144+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:57.493 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:57.488+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:57.880 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:32:57.873+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:32:57.914 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:57.908+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:57.922 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:32:57.913+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 46sec e323 v323) 2019-12-09T15:32:57.947 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:57.940+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:58.041 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:58.035+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:58.110 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:58.104+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:58.504 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:58.498+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:58.866 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:32:58.861+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:32:58.967 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:58.961+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:32:58.981 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:58.976+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:32:59.003 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:32:58.998+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:32:59.112 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:32:59.107+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:32:59.461 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:32:59.456+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:32:59.903 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:32:59.896+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:32:59.940 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:32:59.933+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:33:00.006 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:32:59.999+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:33:00.035 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:00.030+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:33:00.066 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:33:00.061+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:33:00.497 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:33:00.492+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:33:00.917 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:33:00.909+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:33:00.920 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:33:00.914+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:33:00.986 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:33:00.981+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:33:01.044 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:33:01.030+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:33:01.069 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:01.063+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:33:01.482 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:33:01.477+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:33:01.906 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:33:01.901+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:33:01.978 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:33:01.972+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:33:01.983 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:33:01.978+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:33:02.035 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:33:02.030+0000 1694f700 -1 osd.1 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.137526+0000 front 2019-12-09T15:31:44.152235+0000 (oldest deadline 2019-12-09T15:32:25.220073+0000) 2019-12-09T15:33:02.074 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:02.069+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:33:02.505 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:33:02.500+0000 1694f700 -1 osd.4 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.127492+0000 front 2019-12-09T15:31:44.133917+0000 (oldest deadline 2019-12-09T15:32:30.011902+0000) 2019-12-09T15:33:02.947 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:33:02.942+0000 1694f700 -1 osd.3 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:44.288502+0000 front 2019-12-09T15:31:44.285263+0000 (oldest deadline 2019-12-09T15:32:27.159306+0000) 2019-12-09T15:33:02.954 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:33:02.949+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 46sec e323 v323) 2019-12-09T15:33:02.959 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:33:02.953+0000 1694f700 -1 osd.7 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:40.104801+0000 front 2019-12-09T15:31:40.091961+0000 (oldest deadline 2019-12-09T15:32:25.370453+0000) 2019-12-09T15:33:03.052 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:33:03.040+0000 1694f700 -1 osd.6 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.310724+0000 front 2019-12-09T15:31:41.312387+0000 (oldest deadline 2019-12-09T15:32:54.605139+0000) 2019-12-09T15:33:03.067 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:03.061+0000 1694f700 -1 osd.0 323 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:31:41.460462+0000 front 2019-12-09T15:31:41.399816+0000 (oldest deadline 2019-12-09T15:32:25.482082+0000) 2019-12-09T15:33:37.360 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:04:58.819 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:37.381 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:04:58.839 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:37.389 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:04:58.848 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:37.393 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:04:58.875 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:37.397 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:04:58.855 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:37.398 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:04:58.880 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:37.757 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:04:59.239 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:37.995 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:04:59.454 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:38.053 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:04:59.513 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:38.841 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:38.728+0000 1694f700 -1 osd.0 347 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:39.776 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:39.770+0000 1694f700 -1 osd.0 347 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:40.798 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:40.792+0000 1694f700 -1 osd.0 348 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:41.773 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:41.767+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:42.768 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:42.759+0000 1694f700 -1 osd.0 349 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:43.800 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:43.794+0000 1694f700 -1 osd.0 350 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:44.804 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:44.798+0000 1694f700 -1 osd.0 351 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:45.910 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:45.848+0000 1694f700 -1 osd.0 352 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:46.998 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:46.992+0000 1694f700 -1 osd.0 353 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:48.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:47.994+0000 1694f700 -1 osd.0 354 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:48.679 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:33:48.670+0000 38f95700 -1 osd.2 354 build_incremental_map_msg missing incremental map 354 2019-12-09T15:33:48.810 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:33:48.801+0000 3478c700 -1 osd.2 354 build_incremental_map_msg missing incremental map 354 2019-12-09T15:33:48.941 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:33:48.923+0000 36790700 -1 osd.2 354 build_incremental_map_msg missing incremental map 354 2019-12-09T15:33:49.053 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:49.042+0000 1694f700 -1 osd.0 354 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:49.191 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:33:49.184+0000 124b0700 -1 osd.2 354 build_incremental_map_msg missing incremental map 354 2019-12-09T15:33:49.242 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:33:49.227+0000 35f8f700 -1 osd.2 354 build_incremental_map_msg missing incremental map 354 2019-12-09T15:33:49.820 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:33:49.800+0000 32f89700 -1 osd.2 354 build_incremental_map_msg missing incremental map 354 2019-12-09T15:33:50.038 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:50.033+0000 1694f700 -1 osd.0 355 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:51.133 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:51.098+0000 1694f700 -1 osd.0 356 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:52.147 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:52.142+0000 1694f700 -1 osd.0 357 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:53.188 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:53.183+0000 1694f700 -1 osd.0 358 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:33:53.752 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:33:53.747+0000 32f89700 -1 osd.2 357 build_incremental_map_msg missing incremental map 354 2019-12-09T15:33:54.243 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:33:54.237+0000 1694f700 -1 osd.0 358 get_health_metrics reporting 13 slow ops, oldest is osd_op(client.4524.0:110579 7.3 7.48c5cee3 (undecoded) ondisk+retry+write+known_if_redirected e326) 2019-12-09T15:34:13.983 INFO:teuthology.orchestra.run.smithi033.stderr:testSerialization (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:34:51.587 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:51.579+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:51.879 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:51.873+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:52.588 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:52.582+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:52.751 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:52.745+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:52.931 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:52.926+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:53.199 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:34:53.193+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:34:53.543 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:53.537+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:53.558 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:34:53.552+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:34:53.628 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:34:53.620+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:34:53.786 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:53.780+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:53.897 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:53.892+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:54.247 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:34:54.242+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:34:54.568 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:54.561+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:54.582 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:34:54.577+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:34:54.611 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:34:54.606+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:34:54.767 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:54.761+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:54.942 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:54.936+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:55.225 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:34:55.219+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:34:55.546 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:55.540+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:55.600 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:34:55.592+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:34:55.631 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:34:55.625+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:34:55.751 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:55.743+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:55.905 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:55.899+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:56.199 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:34:56.192+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:34:56.569 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:56.563+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:56.597 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:34:56.591+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:34:56.615 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:34:56.609+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:34:56.798 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:56.788+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:56.922 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:56.916+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:57.240 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:34:57.235+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:34:57.597 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:34:57.591+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:34:57.623 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:57.616+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:57.640 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:34:57.635+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:34:57.835 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:57.829+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:57.930 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:57.924+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:58.292 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:34:58.287+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:34:58.581 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:34:58.576+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:34:58.673 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:58.662+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:58.691 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:34:58.686+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:34:58.848 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:58.842+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:58.973 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:58.967+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:34:59.280 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:34:59.275+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:34:59.611 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:34:59.606+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:34:59.684 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:34:59.678+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:34:59.726 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:34:59.716+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:34:59.863 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:34:59.853+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:34:59.965 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:34:59.960+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:00.240 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:00.234+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:00.653 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:00.648+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:00.702 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:00.696+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:00.733 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:00.729+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:00.857 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:00.850+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:00.976 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:00.970+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:01.232 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:01.225+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:01.648 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:01.643+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:01.728 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:01.722+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:01.751 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:01.747+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:01.850 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:01.844+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:01.951 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:01.946+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:02.190 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:02.180+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:02.689 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:02.684+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:02.728 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:02.722+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:02.746 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:02.741+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:02.883 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:02.877+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:02.998 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:02.993+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:03.211 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:03.205+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:03.697 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:03.692+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:03.744 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:03.738+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:03.774 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:03.769+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:03.886 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:03.881+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:04.018 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:04.013+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:04.246 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:04.240+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:04.669 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:04.664+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:04.696 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:04.691+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:04.803 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:04.798+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:04.842 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:04.837+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:05.036 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:05.031+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:05.276 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:05.269+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:05.726 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:05.714+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:05.738 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:05.732+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:05.774 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:05.769+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:05.815 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:05.810+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:06.006 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:06.002+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:06.231 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:06.225+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:06.718 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:06.713+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:06.728 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:06.721+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:06.768 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:06.764+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:06.791 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:06.784+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:06.967 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:06.963+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:07.185 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:07.179+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:07.719 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:07.714+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:07.725 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:07.719+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:07.759 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:07.753+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:07.807 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:07.801+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:07.965 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:07.959+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:08.199 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:08.193+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:08.719 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:08.713+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:08.741 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:08.735+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:08.751 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:08.744+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:08.789 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:08.783+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:08.961 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:08.955+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:09.153 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:09.146+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:09.698 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:09.693+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:09.747 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:09.740+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:09.763 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:09.758+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:09.782 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:09.776+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:09.927 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:09.921+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:10.114 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:10.109+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:10.716 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:10.710+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:10.724 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:10.718+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:10.782 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:10.777+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:10.821 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:10.814+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:10.949 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:10.943+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:11.136 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:11.130+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:11.671 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:11.665+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:11.732 INFO:teuthology.orchestra.run.smithi033.stderr:testStackedOverwrite (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:35:11.749 INFO:teuthology.orchestra.run.smithi033.stderr:testTooLongName (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:35:11.750 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:11.744+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:11.778 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:11.772+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:11.784 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:11.777+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:11.958 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:11.953+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:12.100 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:12.095+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:12.635 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:12.629+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:12.758 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:12.753+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:12.759 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:12.753+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:12.801 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:12.796+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:12.956 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:12.950+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:13.094 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:13.088+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:13.611 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:13.606+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:13.789 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:13.783+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:13.793 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:13.786+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:13.795 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:13.790+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:13.965 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:13.955+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:testZeroByteFile (test.functional.tests.TestFileUTF8) ... ok 2019-12-09T15:35:14.137 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:14.132+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:14.596 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:14.590+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:14.749 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:14.743+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:14.757 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:14.751+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:14.837 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:14.831+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:14.982 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:14.977+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:15.104 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:15.096+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:15.612 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:15.607+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:15.720 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:15.714+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:15.782 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:15.777+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:15.804 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:15.798+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:16.032 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:16.027+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:16.087 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:16.080+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:16.638 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:16.633+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:16.723 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:16.717+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:16.770 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:16.764+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:16.797 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:16.791+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:17.029 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:17.023+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:17.123 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:17.116+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:17.635 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:17.630+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:17.700 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:17.690+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:17.786 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:17.778+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:17.831 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:17.820+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:18.030 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:18.025+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:18.150 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:18.145+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:18.591 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:18.585+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:18.707 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:18.701+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:18.806 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:18.800+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:18.837 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:18.830+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:19.019 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:19.014+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:19.201 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:19.195+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:19.550 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:19.545+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:19.693 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:19.687+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:19.819 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:19.813+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:19.857 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:19.850+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:20.049 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:20.043+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:20.244 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:20.238+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:20.509 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:20.502+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:20.729 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:20.721+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:20.868 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:20.861+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:20.882 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:20.876+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:21.101 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:21.095+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:21.226 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:21.221+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:21.550 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:21.542+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:21.690 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:21.684+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:21.831 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:21.825+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:21.880 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:21.874+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:22.115 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:22.109+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:22.227 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:22.220+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:22.510 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:22.503+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:22.675 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:22.670+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:22.849 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:22.843+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:22.862 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:22.856+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:23.120 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:23.115+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:23.234 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:23.226+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:23.479 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:35:23.473+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e369 v369) 2019-12-09T15:35:23.500 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:23.494+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:23.694 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:23.688+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:23.858 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:23.852+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:23.900 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:23.893+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:24.093 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:24.087+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:24.258 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:24.253+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:24.520 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:24.515+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:24.672 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:24.667+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:24.863 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:24.857+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:24.868 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:24.862+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:25.079 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:25.073+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:25.218 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:25.212+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:25.492 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:25.486+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:25.671 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:25.664+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:25.888 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:25.882+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:25.892 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:25.886+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:26.112 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:26.106+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:26.225 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:26.220+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:26.526 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:26.520+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:26.643 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:26.637+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:26.844 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:26.839+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:26.911 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:26.905+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:27.165 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:27.159+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:27.240 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:27.235+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:27.556 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:27.550+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:27.641 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:27.635+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:27.879 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:27.874+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:27.897 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:27.889+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:28.188 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:28.183+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:28.215 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:28.209+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:28.489 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:35:28.483+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e369 v369) 2019-12-09T15:35:28.605 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:28.599+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:28.668 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:28.662+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:28.868 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:28.862+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:28.943 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:28.938+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:29.199 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:29.193+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:29.229 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:29.224+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:29.587 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:29.582+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:29.705 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:29.699+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:29.910 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:29.905+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:29.969 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:29.963+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:30.193 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:30.188+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:30.195 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:30.189+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:30.582 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:30.575+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:30.699 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:30.690+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:30.939 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:30.932+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:30.956 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:30.950+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:31.188 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:31.182+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:31.205 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:31.198+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:31.578 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:31.572+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:31.688 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:31.683+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:31.938 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:31.932+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:31.988 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:31.983+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:32.171 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:32.164+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:32.184 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:32.179+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:32.552 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:32.546+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:32.678 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:35:32.673+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:05.713822+0000 front 2019-12-09T15:34:05.676297+0000 (oldest deadline 2019-12-09T15:34:50.963439+0000) 2019-12-09T15:35:32.915 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:35:32.909+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.679433+0000 front 2019-12-09T15:34:10.681568+0000 (oldest deadline 2019-12-09T15:34:52.897796+0000) 2019-12-09T15:35:32.963 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:35:32.957+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:06.725646+0000 front 2019-12-09T15:34:06.726642+0000 (oldest deadline 2019-12-09T15:34:51.873446+0000) 2019-12-09T15:35:33.135 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:35:33.128+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.266451+0000 front 2019-12-09T15:34:08.272343+0000 (oldest deadline 2019-12-09T15:34:52.333003+0000) 2019-12-09T15:35:33.160 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:35:33.154+0000 1694f700 -1 osd.4 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:10.070924+0000 front 2019-12-09T15:34:10.068246+0000 (oldest deadline 2019-12-09T15:34:51.746456+0000) 2019-12-09T15:35:33.515 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:35:33.510+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:34:08.028298+0000 front 2019-12-09T15:34:08.010493+0000 (oldest deadline 2019-12-09T15:34:52.590657+0000) 2019-12-09T15:35:33.534 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:35:33.529+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e369 v369) 2019-12-09T15:36:31.060 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:36:31.055+0000 32f89700 -1 osd.2 375 build_incremental_map_msg missing incremental map 374 2019-12-09T15:36:34.734 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:36:34.723+0000 32f89700 -1 osd.2 379 build_incremental_map_msg missing incremental map 374 2019-12-09T15:37:05.053 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:08:26.513 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.065 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:08:26.525 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.070 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:08:26.530 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.075 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:08:26.535 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.076 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:08:26.558 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.082 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:08:26.564 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.380 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:08:26.839 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.389 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:08:26.848 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:05.478 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:08:26.960 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.243 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:08:35.702 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.251 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:08:35.710 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.272 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:08:35.732 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.297 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:08:35.757 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.307 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:08:35.766 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.332 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:08:35.791 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.344 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:08:35.826 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.351 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:08:35.833 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:14.400 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:08:35.881 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:31.014 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:31.007+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:31.223 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:31.217+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:32.055 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:32.048+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:32.201 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:32.196+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:33.015 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:33.008+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:33.064 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:33.058+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:33.160 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:33.155+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:33.244 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:33.238+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:34.033 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:34.024+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:34.067 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:34.062+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:34.147 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:34.142+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:34.242 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:34.238+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:35.052 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:35.046+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:35.070 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:35.065+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:35.139 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:35.132+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:35.209 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:35.204+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:35.370 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:35.365+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:36.024 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:36.018+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:36.054 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:36.049+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:36.159 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:36.152+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:36.197 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:36.192+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:36.394 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:36.389+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:36.611 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:36.605+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:37.045 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:37.039+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:37.101 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:37.095+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:37.117 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:37.112+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:37.219 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:37.214+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:37.442 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:37.436+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:37.623 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:37.617+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:38.079 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:38.074+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:38.096 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:38.090+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:38.096 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:38.089+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:38.211 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:38.206+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:38.492 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:38.486+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:38.596 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:38.591+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:39.113 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:39.107+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:39.116 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:39.110+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:39.141 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:39.134+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:39.235 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:39.230+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:39.360 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:09:00.819 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.375 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:09:00.834 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.382 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:09:00.841 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.393 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:09:00.875 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.398 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:09:00.880 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.399 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:09:00.859 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.436 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:09:00.896 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.461 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:09:00.920 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.467 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:09:00.949 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:37:39.496 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:39.489+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:39.643 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:39.638+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:40.086 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:40.081+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:40.148 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:40.142+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:40.167 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:40.161+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:40.238 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:40.233+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:40.474 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:40.468+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:40.691 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:40.686+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:41.074 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:41.068+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:41.114 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:41.108+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:41.162 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:41.156+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:41.201 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:41.195+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:41.468 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:41.458+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:41.676 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:41.670+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:42.050 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:42.044+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:42.081 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:42.075+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:42.139 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:42.133+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:42.180 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:42.175+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:42.458 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:42.451+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:42.694 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:42.689+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:43.103 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:43.098+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:43.106 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:43.099+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:43.130 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:43.124+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:43.139 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:43.133+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:43.500 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:43.493+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:43.714 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:43.708+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:44.064 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:44.059+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:44.064 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:44.058+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:44.130 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:44.124+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:44.169 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:44.165+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:44.509 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:44.503+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:44.690 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:44.686+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:45.050 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:45.045+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:45.070 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:45.062+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:45.156 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:45.150+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:45.184 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:45.180+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:45.507 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:45.499+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:45.735 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:45.730+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:46.062 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:46.051+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:46.119 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:46.112+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:46.165 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:46.159+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:46.196 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:46.191+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:46.499 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:46.491+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:46.705 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:46.700+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:47.066 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:47.061+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:47.119 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:47.112+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:47.205 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:47.200+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:47.213 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:47.207+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:47.519 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:47.512+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:47.730 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:47.726+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:48.050 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:48.044+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:48.139 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:48.133+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:48.222 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:48.215+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:48.230 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:48.224+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:48.571 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:48.564+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:48.754 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:48.750+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:49.068 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:49.063+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:49.164 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:49.157+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:49.235 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:49.228+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:49.283 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:49.278+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:49.553 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:49.546+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:49.727 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:49.720+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:50.041 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:50.036+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:50.198 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:50.192+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:50.278 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:50.268+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:50.331 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:50.325+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:50.512 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:50.505+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:50.743 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:50.737+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:51.094 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:51.088+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:51.179 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:51.173+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:51.249 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:51.242+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:51.315 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:51.309+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:51.506 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:51.496+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:51.746 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:51.740+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:52.104 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:52.099+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:52.132 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:52.126+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:52.292 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:52.284+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:52.344 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:52.338+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:52.533 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:52.526+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:52.767 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:52.759+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:53.127 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:53.120+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:53.139 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:53.134+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:53.321 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:53.311+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:53.335 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:53.329+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:53.495 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:53.488+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:53.721 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:53.715+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:54.111 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:54.104+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:54.179 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:54.173+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:54.313 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:54.307+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:54.370 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:54.364+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:54.452 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:54.445+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:54.719 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:54.709+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:55.119 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:55.111+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:55.156 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:55.147+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:55.291 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:55.285+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:55.335 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:55.329+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:55.480 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:55.474+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:55.743 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:55.738+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:56.131 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:56.125+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:56.135 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:56.126+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:56.317 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:56.310+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:56.373 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:56.368+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:56.446 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:56.439+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:56.779 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:56.774+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:57.099 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:57.091+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:57.132 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:57.127+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:57.337 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:57.331+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:57.418 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:57.412+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:57.454 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:57.447+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:57.810 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:57.804+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:58.095 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:58.088+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:58.095 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:58.089+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:58.322 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:58.315+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:58.466 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:58.460+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:58.488 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:58.481+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:58.772 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:58.766+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:37:59.079 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:37:59.073+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:37:59.095 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:37:59.088+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:37:59.295 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:37:59.289+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:37:59.457 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:37:59.451+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:37:59.518 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:37:59.512+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:37:59.754 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:37:59.749+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:00.055 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:00.050+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:00.125 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:00.119+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:00.294 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:00.287+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:00.442 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:00.434+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:00.509 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:00.503+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:00.796 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:00.790+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:01.063 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:01.056+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:01.140 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:01.134+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:01.281 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:01.276+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:01.491 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:01.486+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:01.520 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:01.515+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:01.797 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:01.791+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:02.024 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:02.018+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:02.154 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:02.149+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:02.272 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:02.265+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:02.522 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:02.515+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:02.533 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:02.528+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:02.813 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:02.807+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:03.050 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:03.043+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:03.149 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:03.142+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:03.233 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:03.227+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:03.508 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:03.502+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:03.524 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:03.517+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:03.782 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:03.776+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:04.088 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:04.080+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:04.150 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:04.144+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:04.281 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:04.276+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:04.382 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:38:04.377+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 47sec e380 v380) 2019-12-09T15:38:04.507 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:04.501+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:04.531 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:04.525+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:04.799 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:04.794+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:05.113 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:05.107+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:05.126 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:05.120+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:05.255 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:05.249+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:05.503 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:05.497+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:05.503 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:05.497+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:05.816 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:05.811+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:06.129 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:06.124+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:06.147 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:06.141+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:06.296 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:06.290+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:06.498 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:06.493+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:06.509 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:06.503+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:06.862 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:06.853+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:07.143 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:07.137+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:07.172 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:07.167+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:07.331 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:07.325+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:07.519 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:07.514+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:07.531 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:07.525+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:07.853 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:07.847+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:08.114 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:08.109+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:08.194 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:08.187+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:08.339 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:08.333+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:08.540 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:08.534+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:08.553 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:08.547+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:08.809 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:08.804+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:09.157 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:09.151+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:09.194 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:09.188+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:09.371 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:09.365+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:09.394 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:38:09.388+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 47sec e380 v380) 2019-12-09T15:38:09.496 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:09.490+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:09.511 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:09.505+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:09.775 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:09.770+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:10.175 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:10.169+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:10.247 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:10.241+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:10.340 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:10.334+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:10.475 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:10.469+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:10.542 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:10.537+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:10.811 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:10.806+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:11.167 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:11.161+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:11.257 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:11.248+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:11.378 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:11.371+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:11.456 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:11.451+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:11.567 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:11.561+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:11.823 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:11.814+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:12.128 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:12.122+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:12.230 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:12.224+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:12.367 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:12.360+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:12.430 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:12.424+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:12.558 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:12.551+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:12.806 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:12.800+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:13.169 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:13.163+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:13.204 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:13.198+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:13.369 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:13.357+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:13.411 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:13.405+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:13.557 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:13.551+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:13.789 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:38:13.783+0000 1694f700 -1 osd.6 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.796356+0000 front 2019-12-09T15:36:49.793140+0000 (oldest deadline 2019-12-09T15:37:35.664805+0000) 2019-12-09T15:38:14.148 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:38:14.142+0000 1694f700 -1 osd.4 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:45.801408+0000 front 2019-12-09T15:36:45.790660+0000 (oldest deadline 2019-12-09T15:37:30.939425+0000) 2019-12-09T15:38:14.221 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:14.216+0000 1694f700 -1 osd.3 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.758698+0000 front 2019-12-09T15:36:49.760717+0000 (oldest deadline 2019-12-09T15:37:32.770584+0000) 2019-12-09T15:38:14.334 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:38:14.330+0000 1694f700 -1 osd.1 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:44.887562+0000 front 2019-12-09T15:36:44.500372+0000 (oldest deadline 2019-12-09T15:37:30.385094+0000) 2019-12-09T15:38:14.429 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:38:14.418+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 47sec e380 v380) 2019-12-09T15:38:14.455 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:38:14.449+0000 1694f700 -1 osd.7 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:47.977312+0000 front 2019-12-09T15:36:47.905683+0000 (oldest deadline 2019-12-09T15:37:33.153929+0000) 2019-12-09T15:38:14.595 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:38:14.589+0000 1694f700 -1 osd.0 380 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:36:49.501516+0000 front 2019-12-09T15:36:49.504644+0000 (oldest deadline 2019-12-09T15:37:34.604887+0000) 2019-12-09T15:38:15.586 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:15.577+0000 1694f700 -1 osd.3 381 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112540 2.3s0 2.17f88a83 (undecoded) ondisk+read+known_if_redirected e380) 2019-12-09T15:38:16.788 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:38:16.780+0000 1694f700 -1 osd.3 382 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112540 2.3s0 2.17f88a83 (undecoded) ondisk+read+known_if_redirected e380) 2019-12-09T15:39:40.451 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:40.444+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:41.424 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:41.420+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:42.412 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:42.407+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:43.383 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:43.379+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:44.386 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:44.382+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:45.358 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:45.353+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:46.371 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:46.366+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:47.329 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:47.323+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:48.349 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:48.343+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:49.335 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:49.329+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:50.289 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:50.283+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:51.245 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:51.239+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:52.286 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:52.281+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:53.303 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:53.297+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:54.273 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:54.268+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:55.319 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:55.314+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:56.352 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:56.346+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:57.398 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:57.392+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:58.386 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:58.380+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:39:59.349 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:39:59.343+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:00.386 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:00.380+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:01.379 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:01.373+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:02.400 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:02.391+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:03.350 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:03.344+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:04.365 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:04.360+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:05.323 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:05.318+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:05.326 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:05.319+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:06.296 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:06.289+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:06.330 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:06.326+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:06.633 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:06.623+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:06.645 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:06.637+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:07.250 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:07.243+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:07.310 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:07.304+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:07.612 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:07.606+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:07.624 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:07.616+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:08.242 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:08.236+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:08.317 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:08.313+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:08.631 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:08.626+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:08.642 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:08.637+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:08.766 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:08.761+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:08.940 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:08.935+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:09.246 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:09.240+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:09.271 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:09.266+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:09.617 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:09.612+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:09.660 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:09.654+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:09.774 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:09.769+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:09.906 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:09.900+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:10.215 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:10.209+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:10.288 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:10.283+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:10.573 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:10.567+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:10.677 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:10.671+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:10.804 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:10.799+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:10.865 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:10.859+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:11.204 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:11.199+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:11.246 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:11.242+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:11.541 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:11.535+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:11.649 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:11.643+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:11.845 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:11.840+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:11.905 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:11.899+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:12.233 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:12.228+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:12.285 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:12.280+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:12.570 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:12.565+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:12.652 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:12.646+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:12.858 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:12.852+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:12.878 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:12.872+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:13.280 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:13.275+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:13.282 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:13.278+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:13.599 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:13.593+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:13.635 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:13.629+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:13.829 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:13.823+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:13.851 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:13.845+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:14.296 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:14.291+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:14.331 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:14.327+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:14.630 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:14.625+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:14.663 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:14.657+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:14.805 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:14.798+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:14.839 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:14.833+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:15.305 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:15.300+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:15.364 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:15.360+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:15.673 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:15.667+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:15.717 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:15.711+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:15.766 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:15.760+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:15.848 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:15.842+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:16.267 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:16.262+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:16.325 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:16.320+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:16.695 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:16.688+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:16.769 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:16.763+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:16.817 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:16.809+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:16.851 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:16.846+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:17.321 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:17.315+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:17.356 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:17.351+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:17.751 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:17.742+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:17.799 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:17.793+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:17.826 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:17.821+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:17.889 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:17.882+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:18.337 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:18.330+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:18.409 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:18.404+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:18.715 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:18.710+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:18.769 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:18.762+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:18.864 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:18.857+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:18.889 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:18.883+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:19.370 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:19.365+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:19.423 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:19.418+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:19.671 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:19.665+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:19.742 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:19.736+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:19.838 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:19.833+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:19.847 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:19.841+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:20.419 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:20.413+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:20.433 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:20.429+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:20.646 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:20.640+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:20.754 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:20.748+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:20.840 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:20.833+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:20.851 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:20.845+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:21.423 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:21.414+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:21.470 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:21.465+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:21.677 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:21.672+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:21.765 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:21.759+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:21.855 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:21.849+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:21.859 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:21.853+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:22.395 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:22.390+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:22.480 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:22.475+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:22.692 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:22.686+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:22.796 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:22.791+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:22.840 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:22.834+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:22.866 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:22.860+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:23.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:23.357+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:23.449 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:23.445+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:23.698 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:23.694+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:23.772 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:23.766+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:23.801 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:23.791+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:23.881 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:23.876+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:24.382 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:24.376+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:24.462 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:24.457+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:24.463 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:24.458+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:24.750 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:24.744+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:24.771 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:24.766+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:24.819 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:24.813+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:24.918 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:24.912+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:25.412 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:25.407+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:25.512 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:25.506+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:25.513 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:25.508+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:25.784 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:25.779+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:25.786 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:25.781+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:25.788 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:25.782+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:25.892 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:25.885+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:26.412 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:26.406+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:26.468 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:26.462+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:26.470 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:26.465+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:26.754 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:26.744+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:26.756 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:26.751+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:26.839 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:26.833+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:26.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:26.868+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:27.394 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:27.389+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:27.481 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:27.476+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:27.483 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:27.478+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:27.739 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:27.735+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:27.768 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:27.763+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:27.829 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:27.824+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:27.872 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:27.866+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:28.370 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:28.364+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:28.513 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:28.503+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:28.514 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:28.507+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:28.718 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:28.714+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:28.779 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:28.774+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:28.873 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:28.868+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:28.888 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:28.881+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:29.329 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:29.323+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:29.525 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:29.519+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:29.526 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:29.520+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:29.739 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:29.733+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:29.764 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:29.759+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:29.908 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:29.903+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:29.909 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:29.904+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:30.360 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:30.354+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:30.554 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:30.548+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:30.555 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:30.550+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:30.723 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:30.718+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:30.764 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:30.759+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:30.885 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:30.878+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:30.904 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:30.900+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:31.316 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:31.310+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:31.567 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:31.560+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:31.568 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:31.562+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:31.718 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:31.713+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:31.778 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:31.773+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:31.874 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:31.868+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:31.937 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:31.932+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:32.339 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:32.332+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:32.588 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:32.582+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:32.588 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:32.583+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:32.738 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:32.733+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:32.747 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:32.743+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:32.923 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:32.919+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:32.927 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:32.921+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:33.294 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:33.287+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:33.622 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:33.616+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:33.623 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:33.618+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:33.731 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:33.725+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:33.759 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:33.754+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:33.896 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:33.889+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:33.958 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:33.952+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:34.251 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:34.245+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:34.673 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:34.668+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:34.674 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:34.669+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:34.753 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:34.747+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:34.782 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:34.776+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:34.883 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:34.878+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:34.996 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:34.991+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:35.269 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:35.263+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:35.631 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:35.625+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:35.633 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:35.629+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:35.774 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:35.768+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:35.825 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:35.819+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:35.925 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:35.919+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:36.035 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:36.029+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:36.304 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:36.299+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:36.673 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:36.668+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:36.675 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:36.671+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:36.768 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:36.762+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:36.832 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:36.826+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:36.921 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:36.914+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:37.068 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:37.062+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:37.307 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:37.301+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:37.720 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:37.713+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:37.721 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:37.716+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:37.781 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:37.775+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:37.788 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:37.783+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:37.951 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:37.945+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:38.089 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:38.083+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:38.303 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:38.298+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:38.677 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:38.671+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:38.677 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:38.672+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:38.814 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:38.808+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:38.819 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:38.813+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:38.912 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:38.906+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:39.139 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:39.134+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:39.318 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:39.312+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:39.677 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:39.672+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:39.679 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:39.674+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:39.853 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:39.847+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:39.866 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:39.850+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:39.866 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:40:39.861+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 43sec e391 v391) 2019-12-09T15:40:39.891 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:39.885+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:40.104 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:40.099+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:40.321 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:40.315+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:40.676 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:40.669+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:40.678 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:40.673+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:40.816 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:40.810+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:40.835 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:40.830+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:40.943 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:40.937+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:41.128 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:41.123+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:41.281 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:41.276+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:41.692 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:41.684+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:41.698 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:41.687+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:41.798 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:41.793+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:41.810 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:41.805+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:41.955 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:41.949+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:42.170 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:42.164+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:42.272 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:42.267+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:42.699 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:42.693+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:42.701 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:42.697+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:42.849 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:42.843+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:42.858 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:42.853+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:42.980 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:42.971+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:43.167 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:43.161+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:43.265 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:43.259+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:43.689 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:43.681+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:43.690 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:43.685+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:43.829 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:43.823+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:43.852 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:43.846+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:43.946 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:43.938+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:44.140 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:44.134+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:44.271 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:44.265+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:44.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:44.660+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:44.666 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:44.661+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:44.828 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:44.822+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:44.869 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:44.864+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:44.878 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:40:44.872+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 43sec e391 v391) 2019-12-09T15:40:44.941 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:44.936+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:45.114 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:45.108+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:45.279 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:45.274+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:45.655 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:45.649+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:45.656 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:45.651+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:45.851 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:45.846+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:45.855 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:45.849+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:45.968 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:45.961+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:46.110 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:46.105+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:46.270 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:46.265+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:46.665 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:46.660+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:46.667 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:46.663+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:46.814 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:46.808+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:46.891 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:46.883+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:46.967 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:46.961+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:47.116 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:47.110+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:47.314 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:47.309+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:47.717 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:47.711+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:47.719 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:47.714+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:47.778 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:47.772+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:47.850 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:47.844+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:47.994 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:47.987+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:48.149 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:48.143+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:48.341 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:48.335+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:48.751 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:48.746+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:48.752 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:48.745+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:48.754 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:48.747+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:48.814 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:48.807+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:48.992 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:48.985+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:49.197 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:40:49.191+0000 1694f700 -1 osd.4 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.438802+0000 front 2019-12-09T15:39:22.907990+0000 (oldest deadline 2019-12-09T15:40:05.899256+0000) 2019-12-09T15:40:49.350 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:40:49.345+0000 1694f700 -1 osd.3 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:22.111072+0000 front 2019-12-09T15:39:22.137339+0000 (oldest deadline 2019-12-09T15:40:04.968755+0000) 2019-12-09T15:40:49.737 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:49.731+0000 1694f700 -1 osd.1 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.977865+0000 front 2019-12-09T15:39:23.160455+0000 (oldest deadline 2019-12-09T15:40:24.446009+0000) 2019-12-09T15:40:49.738 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:49.733+0000 1694f700 -1 osd.1 391 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:49.746 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:40:49.740+0000 1694f700 -1 osd.7 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:23.191034+0000 front 2019-12-09T15:39:23.986965+0000 (oldest deadline 2019-12-09T15:40:08.395082+0000) 2019-12-09T15:40:49.818 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:40:49.812+0000 1694f700 -1 osd.6 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.371313+0000 front 2019-12-09T15:39:24.333056+0000 (oldest deadline 2019-12-09T15:40:06.551211+0000) 2019-12-09T15:40:49.928 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:40:49.919+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 43sec e391 v391) 2019-12-09T15:40:49.944 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:40:49.939+0000 1694f700 -1 osd.0 391 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:39:24.731112+0000 front 2019-12-09T15:39:24.505898+0000 (oldest deadline 2019-12-09T15:40:07.942795+0000) 2019-12-09T15:40:50.772 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:50.753+0000 1694f700 -1 osd.1 392 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:51.899 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:40:51.861+0000 1694f700 -1 osd.1 393 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:112771 2.33s0 2:cc936f44:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.664_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (343) in=360b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (483) in=504b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e389) 2019-12-09T15:40:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:test_slo_etag_is_hash_of_etags (test.functional.tests.TestSlo) ... ok 2019-12-09T15:40:55.711 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:12:17.171 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:55.720 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:12:17.179 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:55.735 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:12:17.217 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:55.737 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:12:17.197 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:55.744 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:12:17.203 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:55.744 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:12:17.226 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:56.016 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:12:17.498 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:56.017 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:12:17.477 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:40:56.021 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:12:17.481 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:11.658 INFO:teuthology.orchestra.run.smithi033.stderr:test_slo_get_simple_manifest (test.functional.tests.TestSlo) ... ok 2019-12-09T15:41:13.859 INFO:teuthology.orchestra.run.smithi033.stderr:test_slo_ranged_get (test.functional.tests.TestSlo) ... ok 2019-12-09T15:42:15.172 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:15.160+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:16.121 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:16.116+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:17.166 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:17.161+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:18.135 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:18.130+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:19.133 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:19.129+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:20.161 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:20.156+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:21.142 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:21.136+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:22.140 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:22.136+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:23.177 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:23.171+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:24.218 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:24.214+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:25.263 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:25.257+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:26.313 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:26.306+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:27.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:27.357+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:28.394 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:28.389+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:29.348 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:29.343+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:30.386 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:30.381+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:31.407 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:31.399+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:32.398 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:32.393+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:33.408 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:33.403+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:34.361 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:34.356+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:35.369 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:35.365+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:35.845 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:35.839+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:36.022 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:36.016+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:36.325 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:36.320+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:36.704 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:36.697+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:36.856 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:36.851+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:36.869 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:36.864+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:36.886 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:36.879+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:37.041 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:37.034+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:37.334 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:37.330+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:37.730 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:37.724+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:37.848 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:37.843+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:37.885 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:37.880+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:37.901 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:37.893+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:38.011 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:38.004+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:38.293 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:38.286+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:38.294 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:38.288+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:38.701 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:38.695+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:38.822 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:38.814+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:38.866 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:38.860+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:38.869 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:38.863+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:39.001 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:38.995+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:39.341 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:39.334+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:39.341 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:39.336+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:39.760 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:39.747+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:39.831 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:39.825+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:39.869 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:39.863+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:39.878 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:39.872+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:40.012 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:40.007+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:40.357 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:40.349+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:40.359 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:40.352+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:40.747 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:40.741+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:40.853 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:40.845+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:40.853 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:40.842+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:40.917 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:40.912+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:40.973 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:40.967+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:41.348 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:41.340+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:41.348 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:41.343+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:41.789 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:41.783+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:41.817 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:41.811+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:41.857 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:41.851+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:41.880 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:41.874+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:41.987 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:41.977+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:42.396 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:42.390+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:42.397 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:42.392+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:42.841 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:42.832+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:42.863 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:42.858+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:42.906 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:42.900+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:42.909 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:42.903+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:42.964 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:42.959+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:43.354 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:43.348+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:43.354 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:43.350+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:43.806 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:43.801+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:43.862 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:43.855+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:43.892 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:43.886+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:43.930 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:43.924+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:43.999 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:42:43.993+0000 1694f700 -1 osd.0 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.655729+0000 front 2019-12-09T15:41:50.661516+0000 (oldest deadline 2019-12-09T15:42:35.339822+0000) 2019-12-09T15:42:44.333 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:44.328+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:44.334 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:44.329+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:44.775 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:44.769+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:44.880 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:44.873+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:44.893 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:44.887+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:44.899 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:44.894+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:45.314 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:45.308+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:45.314 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:45.310+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:45.738 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:45.733+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:45.885 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:45.878+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:45.890 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:45.885+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:45.892 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:45.886+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:46.319 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:46.313+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:46.319 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:46.314+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:46.791 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:46.786+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:46.852 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:46.846+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:46.870 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:46.863+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:46.881 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:46.875+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:47.299 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:47.292+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:47.300 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:47.295+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:47.821 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:47.815+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:47.865 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:47.860+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:47.873 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:47.867+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:47.915 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:47.905+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:48.278 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:48.273+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:48.279 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:48.274+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:48.845 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:48.839+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:48.854 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:48.849+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:48.889 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:48.884+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:48.902 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:48.897+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:49.299 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:49.293+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:49.299 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:49.294+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:49.872 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:49.867+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:49.874 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:49.864+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:49.904 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:49.898+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:49.922 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:49.917+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:50.350 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:50.344+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:50.350 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:50.345+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:50.859 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:50.852+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:50.878 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:50.873+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:50.898 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:50.892+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:50.917 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:50.912+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:51.388 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:51.382+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:51.388 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:51.384+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:51.857 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:51.851+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:51.893 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:51.885+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:51.910 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:51.904+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:51.944 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:51.938+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:52.362 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:52.357+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:52.363 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:52.358+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:52.825 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:52.815+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:52.907 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:52.901+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:52.937 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:52.931+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:52.970 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:52.964+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:53.347 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:53.341+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:53.348 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:53.342+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:53.808 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:53.802+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:53.897 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:53.892+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:53.975 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:53.969+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:53.998 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:53.993+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:54.386 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:54.379+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:54.387 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:54.381+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:54.854 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:54.847+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:54.866 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:54.860+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:55.015 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:55.007+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:55.050 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:55.045+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:55.429 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:55.422+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:55.430 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:55.423+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:55.861 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:55.854+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:55.902 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:55.896+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:56.032 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:56.027+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:56.058 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:56.052+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:56.432 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:56.425+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:56.433 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:56.426+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:56.913 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:56.906+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:56.934 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:56.929+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:56.998 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:56.992+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:57.022 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:57.017+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:57.438 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:57.428+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:57.439 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:57.429+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:57.908 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:57.902+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:57.956 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:57.947+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:57.981 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:57.975+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:58.063 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:58.056+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:58.473 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:58.463+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:58.474 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:58.465+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:58.897 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:58.892+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:42:58.929 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:58.921+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:59.028 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:42:59.022+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:42:59.087 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:42:59.081+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:42:59.507 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:59.501+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:42:59.508 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:42:59.502+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:42:59.905 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:42:59.898+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:42:59.931 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:42:59.925+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:00.075 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:00.069+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:00.101 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:00.095+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:00.543 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:00.537+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:00.544 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:00.538+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:00.862 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:00.855+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:00.893 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:00.887+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:01.043 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:01.035+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:01.056 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:01.050+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:01.561 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:01.555+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:01.562 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:01.555+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:01.824 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:01.818+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:01.873 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:01.867+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:02.056 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:02.051+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:02.081 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:02.076+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:02.528 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:02.521+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:02.529 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:02.523+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:02.840 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:02.834+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:02.841 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:02.836+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:03.057 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:03.052+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:03.061 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:03.056+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:03.484 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:03.477+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:03.484 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:03.479+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:03.881 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:03.874+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:03.881 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:03.875+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:04.053 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:04.045+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:04.092 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:04.087+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:04.478 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:04.472+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:04.478 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:04.473+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:04.883 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:04.874+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:04.926 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:04.921+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:05.063 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:05.057+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:05.106 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:05.100+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:05.515 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:05.509+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:05.516 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:05.510+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:05.843 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:05.833+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:05.899 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:05.893+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:06.037 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:06.031+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:06.066 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:06.061+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:06.499 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:06.493+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:06.500 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:06.494+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:06.872 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:06.865+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:06.881 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:06.870+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:07.053 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:07.047+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:07.074 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:07.064+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:07.481 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:07.474+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:07.482 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:07.476+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:07.871 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:07.865+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:07.907 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:07.900+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:08.038 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:08.032+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:08.066 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:08.060+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:08.486 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:08.478+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:08.487 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:08.480+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:08.870 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:08.864+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:08.948 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:08.940+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:08.998 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:08.992+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:09.115 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:09.110+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:09.520 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:09.513+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:09.521 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:09.515+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:09.835 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:09.830+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:09.926 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:09.918+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:10.030 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:10.026+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:10.149 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:10.144+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:10.479 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:43:10.471+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e402 v402) 2019-12-09T15:43:10.540 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:10.533+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:10.540 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:10.534+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:10.855 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:10.851+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:10.913 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:10.906+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:11.022 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:11.017+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:11.139 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:11.134+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:11.572 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:11.565+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:11.573 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:11.566+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:11.909 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:11.904+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:11.957 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:11.952+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:11.999 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:11.994+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:12.186 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:12.182+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:12.535 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:12.529+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:12.536 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:12.531+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:12.906 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:12.902+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:12.998 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:12.993+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:13.008 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:13.003+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:13.219 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:13.214+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:13.523 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:13.518+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:13.524 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:13.519+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:13.881 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:13.876+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:14.013 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:14.007+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:14.026 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:14.021+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:14.194 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:14.190+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:14.530 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:14.524+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:14.531 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:14.526+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:14.838 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:43:14.834+0000 1694f700 -1 osd.4 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:50.401035+0000 front 2019-12-09T15:41:50.402824+0000 (oldest deadline 2019-12-09T15:42:36.277932+0000) 2019-12-09T15:43:15.036 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:43:15.032+0000 1694f700 -1 osd.6 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:49.859142+0000 front 2019-12-09T15:41:49.858501+0000 (oldest deadline 2019-12-09T15:42:35.199001+0000) 2019-12-09T15:43:15.078 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:43:15.071+0000 1694f700 -1 osd.1 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:53.661650+0000 front 2019-12-09T15:41:53.696070+0000 (oldest deadline 2019-12-09T15:42:35.948426+0000) 2019-12-09T15:43:15.216 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:43:15.211+0000 1694f700 -1 osd.7 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:51.919218+0000 front 2019-12-09T15:41:51.886807+0000 (oldest deadline 2019-12-09T15:42:36.528725+0000) 2019-12-09T15:43:15.522 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:43:15.516+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e402 v402) 2019-12-09T15:43:15.533 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:15.527+0000 1694f700 -1 osd.3 402 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:41:52.016012+0000 front 2019-12-09T15:41:52.080470+0000 (oldest deadline 2019-12-09T15:42:37.618423+0000) 2019-12-09T15:43:15.533 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:15.528+0000 1694f700 -1 osd.3 402 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:16.725 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:16.709+0000 1694f700 -1 osd.3 403 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:17.790 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:43:17.773+0000 1694f700 -1 osd.3 404 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4524.0:113141 2.bs0 2:d2120dd7:::d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf.4370.665_manifest-abcde:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (356) in=373b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.slo_manifest (548) in=569b,setxattr user.rgw.x-amz-meta-static-large-object (4) in=43b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e402) 2019-12-09T15:43:19.299 INFO:teuthology.orchestra.run.smithi033.stderr:test_slo_etag_is_hash_of_etags (test.functional.tests.TestSloUTF8) ... ok 2019-12-09T15:43:45.954 INFO:teuthology.orchestra.run.smithi033.stderr:test_slo_get_simple_manifest (test.functional.tests.TestSloUTF8) ... ok 2019-12-09T15:43:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:test_slo_ranged_get (test.functional.tests.TestSloUTF8) ... ok 2019-12-09T15:43:47.527 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T15:43:47.528 INFO:teuthology.orchestra.run.smithi033.stderr:---------------------------------------------------------------------- 2019-12-09T15:43:47.528 INFO:teuthology.orchestra.run.smithi033.stderr:Ran 147 tests in 6903.962s 2019-12-09T15:43:47.528 INFO:teuthology.orchestra.run.smithi033.stderr: 2019-12-09T15:43:47.528 INFO:teuthology.orchestra.run.smithi033.stderr:OK 2019-12-09T15:43:47.550 INFO:teuthology.orchestra.run.smithi033:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user rm --uid foo.client.0 --purge-data 2019-12-09T15:43:47.696 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.692+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:47.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.727+0000 7f5c5a286ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T15:43:47.731 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.727+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:47.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.743+0000 7f5c5a286ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T15:43:47.747 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.743+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:47.757 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.753+0000 7f5c5a286ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T15:43:47.757 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.753+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:47.779 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.775+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T15:43:47.973 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.969+0000 7f5c5a286ec0 20 RGWRados::pool_iterate: got zonegroup_info.f120176b-5e2a-495f-add4-cc9bc9689b1d 2019-12-09T15:43:47.973 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.969+0000 7f5c5a286ec0 20 RGWRados::pool_iterate: got zone_info.d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf 2019-12-09T15:43:47.974 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.969+0000 7f5c5a286ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T15:43:47.974 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.969+0000 7f5c5a286ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T15:43:47.974 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.969+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:47.991 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.987+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T15:43:47.991 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:47.987+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.000+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T15:43:48.004 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.000+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.021 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.017+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T15:43:48.021 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.017+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.040 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.036+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T15:43:48.041 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.036+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.045+0000 7f5c5a286ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T15:43:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.045+0000 7f5c5a286ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T15:43:48.049 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.045+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.054+0000 7f5c5a286ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T15:43:48.059 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.054+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.071 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.067+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T15:43:48.071 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.067+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.080+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T15:43:48.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.080+0000 7f5c5a286ec0 20 zonegroup default 2019-12-09T15:43:48.084 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.080+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.089+0000 7f5c5a286ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T15:43:48.093 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.089+0000 7f5c5a286ec0 10 Cannot find current period zone using local zone 2019-12-09T15:43:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.089+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.105 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.098+0000 7f5c5a286ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T15:43:48.106 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.098+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.114 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.110+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T15:43:48.115 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.110+0000 7f5c5a286ec0 20 rados->read ofs=0 len=0 2019-12-09T15:43:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.125+0000 7f5c5a286ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T15:43:48.129 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.125+0000 7f5c5a286ec0 20 zone default found 2019-12-09T15:43:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.125+0000 7f5c5a286ec0 20 started sync module instance, tier type = 2019-12-09T15:43:48.130 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.125+0000 7f5c5a286ec0 20 started zone id=d7b54479-3ea8-4d4d-b4ef-b3b6c4677caf (name=default) with tier type = 2019-12-09T15:43:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.508+0000 7f5c5a286ec0 20 add_watcher() i=0 2019-12-09T15:43:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.508+0000 7f5c5a286ec0 20 add_watcher() i=1 2019-12-09T15:43:48.512 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.508+0000 7f5c5a286ec0 20 add_watcher() i=2 2019-12-09T15:43:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.508+0000 7f5c5a286ec0 20 add_watcher() i=3 2019-12-09T15:43:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.508+0000 7f5c5a286ec0 20 add_watcher() i=4 2019-12-09T15:43:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.508+0000 7f5c5a286ec0 20 add_watcher() i=5 2019-12-09T15:43:48.513 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.508+0000 7f5c5a286ec0 20 add_watcher() i=6 2019-12-09T15:43:48.545 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.541+0000 7f5c5a286ec0 20 add_watcher() i=7 2019-12-09T15:43:48.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.541+0000 7f5c5a286ec0 2 all 8 watchers are set, enabling cache 2019-12-09T15:43:48.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.541+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:43:48.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.555+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3006146966 >> v1:172.21.15.33:6789/0 conn(0x55f3297f0920 legacy=0x55f3297f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T15:43:48.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:43:48.695+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3006146966 >> [v2:172.21.15.186:6810/10132,v1:172.21.15.186:6811/10132] conn(0x55f329802a30 msgr2=0x55f329802e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T15:44:04.173 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:44:04.162+0000 3578e700 -1 osd.2 407 build_incremental_map_msg missing incremental map 407 2019-12-09T15:44:04.228 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:44:04.192+0000 3578e700 -1 osd.2 407 build_incremental_map_msg missing incremental map 407 2019-12-09T15:44:04.243 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:44:04.221+0000 32788700 -1 osd.2 407 build_incremental_map_msg missing incremental map 407 2019-12-09T15:44:05.557 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:44:05.442+0000 32f89700 -1 osd.2 408 build_incremental_map_msg missing incremental map 407 2019-12-09T15:44:05.705 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:44:05.459+0000 32f89700 -1 osd.2 408 build_incremental_map_msg missing incremental map 407 2019-12-09T15:44:08.980 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:44:08.974+0000 32f89700 -1 osd.2 411 build_incremental_map_msg missing incremental map 407 2019-12-09T15:44:10.545 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:44:10.541+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:44:32.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:44:32.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:44:44.419 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:16:05.879 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.431 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:16:05.890 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.436 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:16:05.895 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.444 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:16:05.926 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.445 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:16:05.905 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.448 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:16:05.930 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.737 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:16:06.196 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.784 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:16:06.243 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:44.793 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:16:06.275 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:44:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:44:54.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:45:06.337 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:06.331+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:07.129 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:07.122+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:07.197 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:07.191+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:07.370 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:07.363+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:08.171 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:08.164+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:08.191 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:08.182+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:08.270 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:08.263+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:08.398 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:08.392+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:09.157 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:09.150+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:09.187 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:09.182+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:09.217 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:09.210+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:09.303 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:09.297+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:09.450 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:09.444+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:10.186 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:10.180+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:10.201 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:10.195+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:10.203 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:10.197+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:10.294 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:10.286+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:10.418 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:10.413+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:10.506 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:10.501+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:11.165 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:11.159+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:11.185 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:11.180+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:11.189 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:11.182+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:11.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:11.317+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:11.380 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:11.375+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:11.524 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:11.518+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:12.145 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:12.140+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:12.156 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:12.150+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:12.184 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:12.177+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:12.318 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:12.312+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:12.343 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:12.337+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:12.535 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:12.530+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:13.115 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:13.107+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:13.136 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:13.131+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:13.231 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:13.224+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:13.329 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:13.321+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:13.382 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:13.376+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:13.493 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:13.488+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:14.084 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:14.078+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:14.098 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:14.087+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:14.261 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:14.253+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:14.301 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:14.294+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:14.339 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:14.333+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:14.545 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:14.537+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:15.113 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:15.105+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:15.116 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:15.111+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:15.295 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:15.288+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:15.355 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:15.347+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:15.383 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:15.378+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:15.588 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:15.583+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:16.116 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:16.109+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:16.151 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:16.145+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:16.264 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:16.257+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:16.314 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:16.306+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:16.380 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:16.374+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:16.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:45:16.541+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:45:16.630 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:16.622+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:17.133 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:17.127+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:17.158 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:17.152+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:17.271 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:17.266+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:17.326 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:17.321+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:17.385 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:17.380+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:17.619 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:17.613+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:18.120 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:18.114+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:18.130 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:18.125+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:18.225 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:18.219+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:18.325 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:18.320+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:18.346 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:18.341+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:18.616 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:18.610+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:19.088 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:19.083+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:19.151 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:19.146+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:19.204 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:19.197+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:19.302 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:19.297+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:19.331 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:19.325+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:19.596 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:19.591+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:20.094 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:20.087+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:20.125 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:20.119+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:20.173 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:20.163+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:20.267 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:20.262+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:20.290 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:20.283+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:20.580 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:20.574+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:21.086 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:21.080+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:21.155 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:21.150+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:21.221 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:21.214+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:21.313 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:21.306+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:21.326 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:21.319+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:21.567 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:21.561+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:22.094 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:22.088+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:22.145 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:22.139+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:22.232 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:22.226+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:22.297 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:22.290+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:22.360 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:22.355+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:22.550 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:22.544+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:23.146 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:23.140+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:23.194 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:23.188+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:23.267 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:23.260+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:23.299 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:23.293+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:23.395 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:23.390+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:23.554 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:23.548+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:24.119 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:24.112+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:24.177 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:24.171+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:24.231 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:24.225+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:24.268 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:24.263+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:24.381 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:24.376+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:24.557 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:24.552+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:25.163 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:25.158+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:25.173 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:25.165+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:25.204 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:25.198+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:25.292 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:25.286+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:25.388 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:25.383+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:25.584 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:25.579+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:26.128 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:26.123+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:26.179 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:26.174+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:26.245 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:26.240+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:26.321 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:26.311+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:26.409 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:26.403+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:26.600 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:26.595+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:27.175 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:27.169+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:27.206 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:27.200+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:27.210 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:27.204+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:27.315 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:27.309+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:27.381 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:27.375+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:27.636 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:27.631+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:28.134 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:28.129+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:28.222 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:28.216+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:28.253 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:28.248+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:28.317 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:28.312+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:28.340 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:28.334+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:28.630 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:28.625+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:29.141 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:29.136+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:29.223 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:29.216+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:29.245 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:29.239+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:29.275 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:29.269+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:29.391 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:29.348+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:29.597 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:29.591+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:30.184 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:30.178+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:30.205 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:30.198+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:30.234 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:30.228+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:30.308 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:30.302+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:30.365 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:30.360+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:30.583 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:30.578+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:31.160 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:31.153+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:31.192 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:31.187+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:31.212 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:31.206+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:31.272 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:31.265+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:31.367 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:31.361+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:31.628 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:31.623+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:32.182 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:32.176+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:32.208 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:32.202+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:32.248 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:32.242+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:32.250 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:32.245+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:32.338 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:32.332+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:32.585 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:32.579+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:33.176 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:33.170+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:33.200 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:33.193+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:33.210 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:33.203+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:33.211 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:33.206+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:33.315 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:33.310+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:33.556 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:33.550+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:34.149 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:34.143+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:34.227 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:34.222+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:34.237 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:34.233+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:34.260 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:34.256+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:34.314 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:34.309+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:34.575 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:34.569+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:35.180 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:35.174+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:35.218 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:35.213+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:35.233 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:35.228+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:35.282 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:35.276+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:35.297 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:35.291+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:35.582 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:35.576+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:36.177 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:36.172+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:36.179 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:36.173+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:36.246 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:36.240+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:36.279 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:36.273+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:36.332 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:36.325+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:36.633 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:36.628+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:37.132 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:37.127+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:37.207 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:37.202+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:37.290 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:37.283+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:37.304 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:37.298+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:37.358 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:37.352+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:37.627 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:37.622+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:38.172 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:38.167+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:38.247 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:38.242+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:38.298 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:38.293+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:38.315 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:38.309+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:38.371 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:38.366+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:38.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:45:38.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:45:38.649 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:38.644+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:39.166 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:39.160+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:39.288 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:39.283+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:39.315 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:39.309+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:39.319 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:39.313+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:39.395 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:39.389+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:39.609 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:39.605+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:40.136 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:40.130+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:40.245 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:40.239+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:40.332 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:40.327+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:40.360 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:40.355+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:40.439 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:40.430+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:40.662 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:40.656+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:40.995 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:45:40.990+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e413 v413) 2019-12-09T15:45:41.180 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:41.173+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:41.223 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:41.218+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:41.284 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:41.280+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:41.339 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:41.334+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:41.466 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:41.461+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:41.708 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:41.702+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:42.190 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:42.183+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:42.251 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:42.245+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:42.285 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:42.279+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:42.354 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:42.348+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:42.481 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:42.476+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:42.717 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:42.712+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:43.168 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:43.159+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:43.218 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:43.213+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:43.309 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:43.303+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:43.336 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:43.330+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:43.477 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:43.471+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:43.714 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:43.708+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:44.211 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:44.205+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:44.254 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:44.249+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:44.298 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:44.290+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:44.355 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:44.349+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:44.501 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:44.496+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:44.745 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:44.740+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:45.212 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:45.207+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:45.215 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:45.209+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:45.344 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:45.338+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:45.387 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:45.381+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:45.541 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:45.536+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:45.779 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:45.774+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:46.004 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:45:45.999+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e413 v413) 2019-12-09T15:45:46.192 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:46.181+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:46.233 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:46.228+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:46.374 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:46.367+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:46.395 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:46.388+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:46.533 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:46.527+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:46.794 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:46.785+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:47.205 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:47.199+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:47.214 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:47.208+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:47.360 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:47.354+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:47.412 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:47.406+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:47.545 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:47.539+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:47.768 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:47.763+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:48.163 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:48.158+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:48.249 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:48.243+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:48.334 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:48.328+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:48.465 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:48.450+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:48.511 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:48.506+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:48.741 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:48.735+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:49.160 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:49.150+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:49.291 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:49.285+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:49.319 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:49.313+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:49.450 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:49.445+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:49.555 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:49.548+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:49.757 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:49.751+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:50.201 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:45:50.196+0000 1694f700 -1 osd.1 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.098171+0000 front 2019-12-09T15:44:23.108280+0000 (oldest deadline 2019-12-09T15:45:07.169236+0000) 2019-12-09T15:45:50.267 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:45:50.262+0000 1694f700 -1 osd.4 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.923377+0000 front 2019-12-09T15:44:23.054955+0000 (oldest deadline 2019-12-09T15:45:06.405706+0000) 2019-12-09T15:45:50.350 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:45:50.345+0000 1694f700 -1 osd.3 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.414352+0000 front 2019-12-09T15:44:24.368355+0000 (oldest deadline 2019-12-09T15:45:08.383403+0000) 2019-12-09T15:45:50.498 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:45:50.492+0000 1694f700 -1 osd.0 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:22.411291+0000 front 2019-12-09T15:44:22.970280+0000 (oldest deadline 2019-12-09T15:45:08.255334+0000) 2019-12-09T15:45:50.589 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:45:50.583+0000 1694f700 -1 osd.7 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:23.010089+0000 front 2019-12-09T15:44:22.727748+0000 (oldest deadline 2019-12-09T15:45:06.192638+0000) 2019-12-09T15:45:50.744 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:45:50.739+0000 1694f700 -1 osd.6 413 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:44:24.002036+0000 front 2019-12-09T15:44:23.935875+0000 (oldest deadline 2019-12-09T15:45:09.824622+0000) 2019-12-09T15:45:51.067 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:45:51.055+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 44sec e413 v413) 2019-12-09T15:46:00.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:46:00.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:46:01.235 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:17:22.694 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.272 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:17:22.731 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.301 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:17:22.761 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.305 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:17:22.764 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.305 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:17:22.787 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.311 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:17:22.791 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.365 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:17:22.825 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.411 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:17:22.893 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:01.443 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:17:22.902 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:22.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:46:22.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:46:37.654 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:46:37.504+0000 31786700 -1 osd.2 418 build_incremental_map_msg missing incremental map 418 2019-12-09T15:46:37.691 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:46:37.508+0000 3578e700 -1 osd.2 418 build_incremental_map_msg missing incremental map 418 2019-12-09T15:46:37.717 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:46:37.592+0000 31786700 -1 osd.2 418 build_incremental_map_msg missing incremental map 418 2019-12-09T15:46:37.742 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:46:37.605+0000 31786700 -1 osd.2 418 build_incremental_map_msg missing incremental map 418 2019-12-09T15:46:38.920 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:46:38.839+0000 32f89700 -1 osd.2 418 build_incremental_map_msg missing incremental map 418 2019-12-09T15:46:39.899 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:46:39.734+0000 32f89700 -1 osd.2 419 build_incremental_map_msg missing incremental map 418 2019-12-09T15:46:40.047 INFO:tasks.ceph.osd.2.smithi033.stderr:2019-12-09T15:46:39.856+0000 36f91700 -1 osd.2 419 build_incremental_map_msg missing incremental map 418 2019-12-09T15:46:44.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:46:44.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:47:06.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:47:06.541+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:47:28.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:47:28.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:47:39.181 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:39.175+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:39.249 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:39.244+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:39.301 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:39.290+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:39.325 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:39.318+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:40.079 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:40.073+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:40.177 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:40.170+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:40.244 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:40.238+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:40.338 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:40.332+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:40.368 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:40.362+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:41.092 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:41.085+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:41.208 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:41.202+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:41.292 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:41.287+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:41.326 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:41.321+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:41.339 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:41.334+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:41.362 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:41.356+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:42.126 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:42.123+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:42.169 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:42.163+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:42.294 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:42.289+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:42.343 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:42.339+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:42.361 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:42.357+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:42.377 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:42.371+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:43.091 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:43.086+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:43.129 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:43.123+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:43.292 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:43.287+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:43.336 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:43.331+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:43.372 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:43.367+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:43.407 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:43.403+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:44.077 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:44.073+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:44.163 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:44.157+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:44.300 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:44.296+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:44.347 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:44.342+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:44.349 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:44.341+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:44.398 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:44.393+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:45.108 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:45.103+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:45.162 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:45.155+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:45.277 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:45.273+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:45.323 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:45.318+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:45.374 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:45.368+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:45.393 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:45.388+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:46.119 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:46.115+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:46.125 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:46.119+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:46.299 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:46.295+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:46.323 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:46.319+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:46.399 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:46.395+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:46.421 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:46.416+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:47.098 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:47.093+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:47.117 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:47.111+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:47.310 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:47.305+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:47.317 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:47.313+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:47.411 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:47.407+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:47.430 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:47.424+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:48.057 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:48.053+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:48.166 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:48.160+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:48.307 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:48.302+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:48.312 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:48.306+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:48.394 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:48.389+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:48.410 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:48.405+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:49.042 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:49.037+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:49.169 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:49.164+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:49.291 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:49.284+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:49.298 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:49.293+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:49.370 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:49.365+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:49.451 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:49.443+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:50.015 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:50.010+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:50.137 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:50.131+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:50.265 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:50.260+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:50.286 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:50.281+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:50.351 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:50.345+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:50.472 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:50.466+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:50.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:47:50.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:47:50.990 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:50.985+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:51.151 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:51.145+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:51.286 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:51.280+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:51.297 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:51.291+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:51.312 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:51.306+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:51.428 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:51.422+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:52.007 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:52.002+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:52.143 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:52.136+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:52.293 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:52.288+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:52.331 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:52.325+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:52.346 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:52.341+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:52.470 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:52.464+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:53.057 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:53.052+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:53.097 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:53.091+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:53.295 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:53.288+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:53.333 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:53.327+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:53.349 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:53.343+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:53.464 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:53.458+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:54.100 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:54.094+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:54.147 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:54.140+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:54.279 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:54.274+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:54.332 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:54.325+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:54.345 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:54.340+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:54.504 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:54.499+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:55.103 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:55.095+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:55.138 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:55.133+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:55.256 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:55.250+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:55.330 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:55.325+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:55.392 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:55.386+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:55.554 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:55.549+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:56.065 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:56.058+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:56.127 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:56.122+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:56.248 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:56.241+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:56.369 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:56.364+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:56.431 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:56.425+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:56.562 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:56.558+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:57.027 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:57.019+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:57.135 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:57.130+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:57.251 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:57.246+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:57.409 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:57.404+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:57.430 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:57.425+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:57.584 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:57.579+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:58.067 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:58.062+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:58.163 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:58.158+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:58.211 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:58.203+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:58.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:58.438+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:58.447 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:58.441+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:58.636 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:58.629+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:47:59.113 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:47:59.107+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:47:59.189 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:47:59.181+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:47:59.225 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:47:59.219+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:47:59.413 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:47:59.403+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:47:59.434 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:47:59.430+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:47:59.593 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:47:59.587+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:00.117 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:00.111+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:00.201 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:00.195+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:00.228 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:00.223+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:00.445 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:00.440+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:00.459 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:00.453+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:00.636 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:00.630+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:01.097 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:01.091+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:01.166 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:01.160+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:01.287 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:01.282+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:01.404 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:01.399+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:01.453 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:01.447+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:01.664 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:01.658+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:02.139 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:02.132+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:02.197 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:02.190+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:02.255 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:02.251+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:02.435 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:02.430+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:02.476 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:02.471+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:02.682 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:02.677+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:03.115 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:03.107+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:03.199 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:03.193+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:03.236 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:03.232+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:03.452 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:03.447+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:03.488 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:03.482+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:03.685 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:03.679+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:04.147 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:04.141+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:04.228 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:04.222+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:04.236 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:04.230+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:04.458 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:04.452+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:04.460 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:04.455+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:04.673 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:04.667+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:05.195 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:05.190+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:05.218 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:05.213+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:05.279 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:05.274+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:05.469 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:05.461+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:05.505 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:05.500+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:05.713 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:05.696+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:06.185 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:06.179+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:06.238 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:06.233+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:06.323 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:06.316+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:06.516 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:06.509+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:06.540 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:06.536+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:06.692 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:06.687+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:07.163 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:07.156+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:07.256 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:07.250+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:07.357 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:07.350+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:07.495 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:07.485+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:07.600 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:07.592+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:07.689 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:07.683+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:08.129 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:08.120+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:08.240 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:08.235+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:08.395 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:08.389+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:08.507 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:08.502+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:08.569 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:08.564+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:08.668 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:08.661+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:09.170 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:09.161+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:09.206 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:09.200+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:09.385 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:09.378+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:09.467 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:09.462+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:09.543 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:09.537+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:09.680 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:09.673+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:10.134 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:10.126+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:10.198 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:10.192+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:10.340 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:10.333+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:10.503 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:10.497+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:10.556 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:10.551+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:10.690 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:10.682+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:11.165 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:11.158+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:11.221 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:11.210+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:11.340 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:11.334+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:11.478 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:11.473+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:11.604 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:11.598+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:11.630 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:48:11.624+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e438 v438) 2019-12-09T15:48:11.712 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:11.706+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:12.150 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:12.142+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:12.186 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:12.181+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:12.353 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:12.342+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:12.459 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:12.453+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:12.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:48:12.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:48:12.622 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:12.610+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:12.707 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:12.701+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:13.194 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:13.186+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:13.211 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:13.206+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:13.375 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:13.362+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:13.495 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:13.490+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:13.645 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:13.640+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:13.741 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:13.735+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:14.223 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:14.217+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:14.251 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:14.245+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:14.416 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:14.410+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:14.533 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:14.528+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:14.668 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:14.663+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:14.715 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:14.708+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:15.197 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:15.191+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:15.285 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:15.280+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:15.445 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:15.438+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:15.566 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:15.560+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:15.673 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:15.668+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:15.763 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:15.755+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:16.195 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:16.188+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:16.286 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:16.281+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:16.433 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:16.425+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:16.591 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:16.586+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:16.646 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:48:16.641+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e438 v438) 2019-12-09T15:48:16.700 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:16.695+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:16.746 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:16.740+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:17.176 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:17.167+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:17.280 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:17.275+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:17.392 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:17.385+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:17.582 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:17.576+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:17.729 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:17.722+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:17.792 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:17.787+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:18.223 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:18.214+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:18.282 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:18.276+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:18.434 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:18.417+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:18.578 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:18.572+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:18.709 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:18.704+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:18.779 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:18.772+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:19.232 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:19.224+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:19.313 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:19.307+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:19.408 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:19.402+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:19.561 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:19.555+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:19.690 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:19.684+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:19.757 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:19.750+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:20.230 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:20.222+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:20.364 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:20.358+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:20.446 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:20.439+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:20.578 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:20.573+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:20.737 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:20.731+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:20.762 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:20.755+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:21.202 INFO:tasks.ceph.osd.0.smithi033.stderr:2019-12-09T15:48:21.194+0000 1694f700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:52.846530+0000 front 2019-12-09T15:46:53.113758+0000 (oldest deadline 2019-12-09T15:47:38.698561+0000) 2019-12-09T15:48:21.387 INFO:tasks.ceph.osd.7.smithi186.stderr:2019-12-09T15:48:21.382+0000 1694f700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:54.478892+0000 front 2019-12-09T15:46:54.546712+0000 (oldest deadline 2019-12-09T15:47:39.147763+0000) 2019-12-09T15:48:21.403 INFO:tasks.ceph.osd.1.smithi033.stderr:2019-12-09T15:48:21.395+0000 1694f700 -1 osd.1 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.985193+0000 front 2019-12-09T15:46:53.567843+0000 (oldest deadline 2019-12-09T15:47:38.782325+0000) 2019-12-09T15:48:21.603 INFO:tasks.ceph.osd.6.smithi186.stderr:2019-12-09T15:48:21.598+0000 1694f700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:53.389318+0000 front 2019-12-09T15:46:53.528977+0000 (oldest deadline 2019-12-09T15:47:38.476408+0000) 2019-12-09T15:48:21.716 INFO:tasks.ceph.mon.a.smithi033.stderr:2019-12-09T15:48:21.707+0000 1a89c700 -1 mon.a@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.2 [v2:172.21.15.33:6812/10163,v1:172.21.15.33:6814/10163] for 45sec e438 v438) 2019-12-09T15:48:21.721 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:48:21.715+0000 1694f700 -1 osd.4 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:55.102958+0000 front 2019-12-09T15:46:55.111889+0000 (oldest deadline 2019-12-09T15:47:38.393114+0000) 2019-12-09T15:48:21.772 INFO:tasks.ceph.osd.3.smithi033.stderr:2019-12-09T15:48:21.765+0000 1694f700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.33:6821 osd.2 since back 2019-12-09T15:46:56.370541+0000 front 2019-12-09T15:46:56.381230+0000 (oldest deadline 2019-12-09T15:47:40.458709+0000) 2019-12-09T15:48:23.158 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:19:44.617 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.168 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:19:44.627 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.178 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:19:44.659 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.182 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:19:44.664 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.196 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:19:44.654 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.198 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:19:44.658 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.388 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:19:44.870 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.508 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:19:44.967 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:23.578 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:19:45.038 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:48:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:48:34.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:48:56.546 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:48:56.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:49:14.225 INFO:tasks.ceph.osd.2.smithi033.stderr:*** Caught signal (Aborted) ** 2019-12-09T15:49:14.228 INFO:tasks.ceph.osd.2.smithi033.stderr: in thread 32f89700 thread_name:tp_osd_tp 2019-12-09T15:49:18.097 INFO:tasks.ceph.osd.2.smithi033.stderr:daemon-helper: command crashed with signal 6 2019-12-09T15:49:18.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:49:18.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:49:40.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:50:02.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:50:02.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:50:24.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:50:24.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:50:46.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:50:46.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:50:47.473 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:22:08.933 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.516 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:22:08.976 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.536 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:22:08.996 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.544 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:22:09.002 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.601 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:22:09.083 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.604 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:22:09.086 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.658 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:22:09.117 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.688 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:22:09.147 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:47.763 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:22:09.245 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:51:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:51:08.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:51:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:51:30.542+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:51:52.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:51:52.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:52:14.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:52:14.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:52:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:52:36.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:52:58.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:52:58.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:53:20.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:53:20.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:53:26.908 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:24:48.367 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:26.914 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:24:48.373 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:26.921 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:24:48.381 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:26.923 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:24:48.405 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:26.927 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:24:48.387 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:26.934 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:24:48.416 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:27.047 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:24:48.506 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:27.112 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:24:48.567 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:27.112 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:24:48.594 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:53:42.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:53:42.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:54:04.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:54:04.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:54:23.373 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:25:44.832 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.416 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:25:44.876 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.455 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:25:44.937 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.461 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:25:44.943 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.461 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:25:44.921 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.463 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:25:44.923 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.472 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:25:44.931 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.524 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:25:44.983 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:23.535 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:25:45.017 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:54:26.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:54:26.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:54:48.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:54:48.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:55:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:55:10.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:55:32.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:55:32.543+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:55:54.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:55:54.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:56:16.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:56:16.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:56:38.548 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:56:38.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:57:00.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:57:00.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:57:22.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:57:22.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:57:44.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:57:44.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:58:06.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:58:06.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:58:28.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:58:28.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:58:39.238 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:30:00.698 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.246 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:30:00.705 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.262 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:30:00.722 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.265 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:30:00.725 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.270 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:30:00.752 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.294 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:30:00.776 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.503 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:30:00.962 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.523 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:30:00.982 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:39.572 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:30:01.054 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:50.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:58:50.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:59:12.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:59:12.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:59:34.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:59:34.544+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T15:59:35.095 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:59:34.984+0000 1694f700 -1 osd.4 465 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.5133.0:1521 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 94502861514784 gen 5] snapc 0=[] ondisk+write+known_if_redirected e441) 2019-12-09T15:59:36.088 INFO:tasks.ceph.osd.4.smithi186.stderr:2019-12-09T15:59:36.083+0000 1694f700 -1 osd.4 466 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.5133.0:1521 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 94502861514784 gen 5] snapc 0=[] ondisk+write+known_if_redirected e441) 2019-12-09T15:59:56.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T15:59:56.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:00:18.549 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:00:18.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:00:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:00:40.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:01:02.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:01:02.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:01:24.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:01:24.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:01:46.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:01:46.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:02:08.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:02:08.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:02:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:02:30.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:02:45.231 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:34:06.690 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.237 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:34:06.697 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.283 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:34:06.742 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.284 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:34:06.744 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.322 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:34:06.804 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.328 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:34:06.810 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.449 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:34:06.909 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.477 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:34:06.936 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:45.568 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:34:07.050 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.466 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:34:08.925 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.469 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:34:08.929 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.476 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:34:08.958 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.503 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:34:08.983 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.513 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:34:08.972 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.592 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:34:09.074 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.707 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:34:09.167 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.708 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:34:09.168 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:47.726 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:34:09.185 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:02:52.545+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:03:14.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:03:14.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:03:36.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:03:36.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:03:58.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:03:58.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:04:20.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:04:20.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:04:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:04:42.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:05:04.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:05:04.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:05:26.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:05:26.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:05:48.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:05:48.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:06:10.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:06:10.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:06:32.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:06:32.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:06:54.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:06:54.546+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:07:16.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:07:16.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:07:38.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:07:38.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:08:00.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:08:00.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:08:01.402 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:39:22.862 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.405 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:39:22.865 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.425 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:39:22.884 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.429 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:39:22.911 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.430 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:39:22.890 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.444 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:39:22.926 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.694 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:39:23.176 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.696 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:39:23.156 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:01.731 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:39:23.191 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:08:22.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:08:22.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:08:44.551 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:08:44.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:09:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:09:06.547+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:09:28.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:09:28.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:09:43.011 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:09:43.007+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c28040150 msgr2=0x7f5c28042570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T16:09:50.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:09:50.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:09:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:09:57.979+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c280347d0 msgr2=0x7f5c280248f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T16:10:03.003 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:10:02.999+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6810/10162,v1:172.21.15.33:6811/10162] conn(0x7f5c28031410 msgr2=0x7f5c28031820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T16:10:12.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:10:12.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:10:34.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:10:34.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:10:56.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:10:56.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:11:06.919 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:42:28.379 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:06.959 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:42:28.419 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:06.969 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:42:28.429 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:06.992 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:42:28.452 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:07.007 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:42:28.488 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:07.017 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:42:28.499 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:07.061 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:42:28.521 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:07.075 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:42:28.535 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:07.145 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:42:28.627 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:11:18.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:11:18.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:11:40.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:11:40.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:12:02.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:12:02.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:12:24.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:12:24.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:12:46.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:12:46.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:12:53.054 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:12:53.050+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c280396d0 msgr2=0x7f5c28039ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T16:13:08.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:13:08.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:13:12.378 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:44:33.838 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.385 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:44:33.845 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.396 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:44:33.856 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.401 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:44:33.861 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.401 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:44:33.883 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.408 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:44:33.891 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.478 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:44:33.938 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.499 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:44:33.958 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:12.557 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:44:34.039 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:13:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:13:30.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:13:52.552 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:13:52.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:14:14.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:14:14.548+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:14:36.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:14:36.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:14:58.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:14:58.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:15:20.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:15:20.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:15:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:15:42.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:16:04.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:16:04.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:16:26.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:16:26.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:16:48.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:16:48.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:17:10.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:17:10.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:17:32.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:17:32.549+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:17:54.553 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:17:54.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:18:16.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:18:16.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:18:25.446 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:49:46.906 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.453 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:49:46.910 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.462 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:49:46.922 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.467 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:49:46.927 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.474 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:49:46.953 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.485 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:49:46.967 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.647 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:49:47.108 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.659 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:49:47.118 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:25.764 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:49:47.246 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:38.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:18:38.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:19:00.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:19:00.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:19:22.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:19:22.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:19:28.614 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:50:50.074 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.649 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:50:50.103 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.649 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:50:50.103 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.661 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:50:50.121 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.697 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:50:50.157 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.708 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:50:50.190 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.711 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:50:50.194 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.719 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:50:50.176 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:28.777 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:50:50.259 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:19:44.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:19:44.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:20:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:20:06.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:20:28.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:20:28.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:20:50.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:20:50.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:21:12.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:21:12.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:21:34.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:21:34.550+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:21:56.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:21:56.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:22:18.554 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:22:18.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:22:40.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:22:40.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:23:02.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:23:02.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:23:24.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:23:24.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:23:36.763 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:54:58.224 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:36.768 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:54:58.229 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:36.786 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:54:58.246 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:36.790 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:54:58.250 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:36.792 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:54:58.275 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:36.799 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:54:58.282 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:36.935 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:54:58.396 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:36.984 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:54:58.444 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:37.082 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:54:58.564 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:46.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:23:46.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:24:08.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:24:08.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:24:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:24:30.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:24:52.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:24:52.551+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:25:14.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:25:14.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:25:36.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:25:36.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:25:58.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:25:58.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:26:20.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:26:20.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:26:42.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:26:42.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:27:04.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:27:04.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:27:26.555 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:27:26.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:27:40.686 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:59:02.142 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:40.718 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:59:02.179 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:40.733 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:59:02.193 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:40.757 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:59:02.217 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:40.798 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:59:02.280 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:40.801 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:59:02.283 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:40.889 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:59:02.350 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:40.912 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:59:02.372 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:41.119 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:59:02.600 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:48.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:27:48.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:27:50.755 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:59:12.215 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.772 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:59:12.232 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.803 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:59:12.263 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.806 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:06:59:12.267 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.814 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:59:12.275 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.834 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:06:59:12.294 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.852 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:59:12.335 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.861 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:59:12.343 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:50.886 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:06:59:12.369 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:10.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:28:10.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:28:32.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:28:32.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:28:48.896 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:00:10.356 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:48.904 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:00:10.365 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:48.920 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:00:10.381 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:48.927 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:00:10.388 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:48.929 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:00:10.412 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:48.939 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:00:10.421 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:49.000 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:00:10.460 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:49.012 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:00:10.472 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:49.051 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:00:10.533 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:54.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:28:54.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:29:08.238+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c28031410 msgr2=0x7f5c28031820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T16:29:16.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:29:16.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:29:38.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:29:38.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:30:00.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:30:00.552+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:30:22.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:30:22.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:30:44.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:30:44.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:31:06.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:31:06.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:31:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:31:28.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:31:50.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:31:50.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:32:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:32:12.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:32:34.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:32:34.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:32:56.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:32:56.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:33:18.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:33:18.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:33:40.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:33:40.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:34:01.130 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:05:22.591 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.138 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:05:22.599 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.157 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:05:22.639 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.159 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:05:22.620 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.163 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:05:22.624 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.166 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:05:22.649 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.363 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:05:22.824 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.369 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:05:22.830 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:01.434 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:05:22.916 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:34:02.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:34:02.553+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:34:24.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:34:24.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:34:46.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:34:46.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:35:08.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:35:08.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:35:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:35:30.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:35:52.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:35:52.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:36:12.481 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:07:33.942 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.508 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:07:33.969 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.561 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:07:34.044 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.564 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:07:34.025 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.565 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:07:34.047 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.566 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:07:34.027 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.636 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:07:34.097 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.642 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:07:34.103 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:12.679 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:07:34.161 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:36:14.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:36:14.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:36:36.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:36:36.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:36:58.557 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:36:58.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:37:20.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:37:20.554+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:37:42.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:37:42.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:38:04.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:38:04.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:38:26.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:38:26.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:38:48.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:38:48.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:39:10.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:39:10.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:39:13.187 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:10:34.648 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.192 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:10:34.654 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.203 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:10:34.665 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.208 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:10:34.669 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.210 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:10:34.693 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.214 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:10:34.696 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.356 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:10:34.817 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.364 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:10:34.825 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:13.395 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:10:34.878 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:32.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:39:32.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:39:54.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:39:54.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:40:16.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:40:16.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:40:38.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:40:38.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:41:00.558 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:41:00.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:41:22.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:41:22.555+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:41:44.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:41:44.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:42:06.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:42:06.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:42:28.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:42:28.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:42:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:42:50.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:43:03.392 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:43:03.387+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c28031410 msgr2=0x7f5c28031820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T16:43:12.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:43:12.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:43:34.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:43:34.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:43:56.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:43:56.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:44:18.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:44:18.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:44:24.099 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:15:45.560 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.112 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:15:45.574 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.119 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:15:45.602 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.128 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:15:45.611 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.138 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:15:45.593 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.138 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:15:45.600 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.315 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:15:45.772 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.357 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:15:45.819 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:24.362 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:15:45.845 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.134 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:15:55.595 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.159 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:15:55.621 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.177 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:15:55.639 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.180 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:15:55.641 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.202 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:15:55.664 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.205 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:15:55.667 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.221 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:15:55.704 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.232 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:15:55.715 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:34.264 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:15:55.747 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:44:40.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:44:40.556+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:45:02.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:45:02.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:45:24.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:45:24.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:45:46.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:45:46.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:46:08.559 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:46:08.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:46:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:46:30.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:46:52.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:46:52.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:46:53.401 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:46:53.399+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c28030ac0 msgr2=0x7f5c28030ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T16:47:14.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:47:14.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:47:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:47:36.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:47:58.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:47:58.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:48:20.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:48:20.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:48:42.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:48:42.557+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:49:04.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:49:04.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:49:26.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:49:26.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:49:35.244 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:20:56.706 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.255 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:20:56.717 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.273 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:20:56.735 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.279 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:20:56.740 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.285 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:20:56.768 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.292 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:20:56.775 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.515 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:20:56.976 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.528 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:20:56.989 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:35.556 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:20:57.039 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:48.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:49:48.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:50:10.560 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:50:10.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:50:32.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:50:32.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:50:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:50:54.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:51:16.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:51:16.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:51:38.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:51:38.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:52:00.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:52:00.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:52:22.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:52:22.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:52:44.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:52:44.558+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:52:55.847 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:24:17.309 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:55.869 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:24:17.331 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:55.919 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:24:17.381 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:55.921 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:24:17.383 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:55.965 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:24:17.448 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:55.968 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:24:17.451 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:55.987 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:24:17.449 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:56.065 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:24:17.527 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:52:56.105 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:24:17.588 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:53:06.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:53:06.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:53:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:53:23.490+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c2800c5d0 msgr2=0x7f5c2800e9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T16:53:28.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:53:28.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:53:50.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:53:50.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:54:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:54:12.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:54:34.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:54:34.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:54:45.327 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:26:06.788 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.342 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:26:06.802 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.349 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:26:06.806 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.349 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:26:06.809 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.362 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:26:06.844 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.368 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:26:06.851 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.443 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:26:06.889 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.447 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:26:06.907 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:45.562 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:26:07.045 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:56.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:54:56.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:55:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:55:18.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:55:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:55:40.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:56:02.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:56:02.559+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:56:24.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:56:24.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:56:46.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:56:46.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:57:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:57:08.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:57:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:57:30.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:57:52.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:57:52.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:58:14.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:58:14.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:58:36.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:58:36.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:58:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:58:58.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:59:20.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:59:20.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:59:42.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T16:59:42.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T16:59:57.439 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:31:18.901 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.448 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:31:18.911 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.459 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:31:18.921 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.463 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:31:18.946 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.468 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:31:18.931 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.476 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:31:18.959 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.648 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:31:19.110 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.658 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:31:19.120 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:57.782 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:31:19.265 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:00:04.562 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:00:04.560+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:00:26.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:00:26.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:00:48.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:00:48.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:01:10.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:01:10.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:01:17.695 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:32:39.157 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.710 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:32:39.173 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.779 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:32:39.241 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.781 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:32:39.244 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.785 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:32:39.268 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.788 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:32:39.271 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.822 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:32:39.284 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.853 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:32:39.315 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:17.890 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:32:39.373 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:01:32.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:01:32.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:01:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:01:54.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:02:16.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:02:16.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:02:38.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:02:38.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:03:00.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:03:00.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:03:22.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:03:22.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:03:44.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:03:44.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:04:06.563 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:04:06.561+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:04:28.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:04:28.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:04:50.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:04:50.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:05:06.819 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:36:28.281 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:06.825 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:36:28.288 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:06.835 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:36:28.297 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:06.839 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:36:28.302 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:06.856 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:36:28.340 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:06.879 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:36:28.360 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:06.991 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:36:28.453 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:07.002 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:36:28.465 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:07.122 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:36:28.606 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:05:12.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:05:34.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:05:34.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:05:56.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:05:56.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:06:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:06:18.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:06:38.639 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:06:38.638+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6826/10160,v1:172.21.15.33:6827/10160] conn(0x7f5c28033e50 msgr2=0x7f5c28031e40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:06:40.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:06:40.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:07:02.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:07:02.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:07:24.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:07:24.562+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:07:46.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:07:46.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:08:08.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:08:08.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:08:09.397 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:39:30.859 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.404 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:39:30.866 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.475 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:39:30.938 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.476 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:39:30.959 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.477 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:39:30.939 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.479 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:39:30.963 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.545 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:39:31.008 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.604 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:39:31.066 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:09.672 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:39:31.155 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:08:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:08:30.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:08:52.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:08:52.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:09:14.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:09:36.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:09:39.777 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:41:01.237 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.782 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:41:01.245 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.791 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:41:01.253 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.825 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:41:01.288 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.829 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:41:01.313 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.834 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:41:01.317 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.871 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:41:01.334 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.913 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:41:01.376 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:39.914 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:41:01.397 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:09:58.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:10:16.027 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:41:37.489 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.037 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:41:37.499 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.047 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:41:37.505 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.048 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:41:37.530 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.048 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:41:37.508 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.055 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:41:37.539 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.085 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:41:37.548 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.106 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:41:37.569 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:16.108 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:41:37.592 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:10:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:10:20.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:10:42.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:10:42.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:11:04.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:11:04.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:11:26.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:11:26.563+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:11:48.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:11:48.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:12:10.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:12:10.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:12:32.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:12:32.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:12:54.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:12:54.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:13:16.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:13:16.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:13:38.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:13:38.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:14:00.565 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:14:00.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:14:22.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:14:22.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:14:44.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:14:44.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:15:06.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:15:06.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:15:28.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:15:28.564+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:15:28.650 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:46:50.113 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.656 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:46:50.119 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.665 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:46:50.128 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.673 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:46:50.136 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.675 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:46:50.159 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.685 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:46:50.169 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.868 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:46:50.331 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.872 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:46:50.335 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:28.946 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:46:50.430 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:50.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:15:50.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:16:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:16:12.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:16:34.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:16:34.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:16:56.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:16:56.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:17:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:17:18.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:17:40.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:17:40.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:18:01.441 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:49:22.902 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.475 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:49:22.938 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.515 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:49:22.978 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.518 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:49:22.982 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.542 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:49:23.026 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.544 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:49:23.028 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.563 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:49:23.026 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.630 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:49:23.093 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:01.630 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:49:23.114 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:18:02.566 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:18:02.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:18:24.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:18:24.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:18:46.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:18:46.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:19:08.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:19:08.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:19:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:19:30.565+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:19:52.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:19:52.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:20:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:20:14.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:20:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:20:36.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:20:38.821 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:52:00.284 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:38.828 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:52:00.291 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:38.840 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:52:00.302 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:38.842 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:52:00.306 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:38.851 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:52:00.335 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:38.883 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:52:00.366 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:38.946 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:52:00.409 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:38.985 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:52:00.448 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:39.090 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:52:00.574 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:20:58.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:21:20.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:21:20.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:21:42.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:21:42.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:22:04.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:22:04.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:22:26.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:22:26.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:22:48.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:22:48.566+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:23:10.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:23:10.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:23:32.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:23:32.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:23:54.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:23:54.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:24:16.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:24:16.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:24:38.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:24:38.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:25:00.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:25:00.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:25:22.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:25:22.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:25:44.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:25:44.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:25:51.057 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:57:12.520 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.074 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:57:12.534 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.076 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:57:12.539 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.079 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:57:12.563 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.082 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:57:12.545 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.090 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:57:12.574 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.280 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:57:12.743 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.291 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:57:12.754 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:51.334 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:57:12.818 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:06.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:26:06.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:26:23.285 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:57:44.748 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.293 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:57:44.756 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.327 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:07:57:44.791 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.348 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:57:44.810 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.350 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:57:44.813 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.350 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:57:44.833 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.352 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:57:44.836 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.392 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:07:57:44.855 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:23.419 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:07:57:44.895 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:26:28.568 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:26:28.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:26:50.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:26:50.567+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:27:12.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:27:12.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:27:34.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:27:34.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:27:56.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:27:56.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:28:18.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:28:18.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:28:40.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:28:40.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:29:02.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:29:02.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:29:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:29:24.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:29:46.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:29:46.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:30:08.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:30:08.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:30:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:30:30.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:30:52.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:30:52.568+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:30:59.327 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:02:20.790 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.332 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:02:20.796 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.343 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:02:20.805 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.346 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:02:20.809 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.362 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:02:20.846 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.368 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:02:20.852 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.537 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:02:21.000 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.551 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:02:21.014 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:59.605 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:02:21.089 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:31:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:31:14.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:31:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:31:36.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:31:58.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:31:58.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:32:20.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:32:20.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:32:42.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:32:42.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:33:04.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:33:04.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:33:26.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:33:26.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:33:48.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:33:48.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:34:10.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:34:10.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:34:32.570 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:34:32.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:34:45.136 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:06:06.599 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.139 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:06:06.602 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.170 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:06:06.633 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.213 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:06:06.677 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.232 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:06:06.716 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.246 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:06:06.729 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.306 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:06:06.770 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.325 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:06:06.788 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:45.464 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:06:06.948 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:34:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:34:54.569+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:35:13.960 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:35:13.960+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c28040150 msgr2=0x7f5c28042570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:35:16.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:35:16.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:35:38.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:35:38.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:36:00.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:36:00.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:36:11.562 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:07:33.016 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.563 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:07:33.022 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.569 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:07:33.033 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.572 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:07:33.035 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.587 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:07:33.071 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.602 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:07:33.086 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.641 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:07:33.104 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.647 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:07:33.110 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:11.722 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:07:33.206 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:36:22.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:36:22.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:36:44.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:36:44.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:37:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:37:06.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:37:28.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:37:28.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:37:50.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:37:50.570+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:38:12.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:38:12.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:38:34.571 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:38:34.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:38:56.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:38:56.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:39:18.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:39:18.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:39:40.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:39:40.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:40:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:40:02.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:40:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:40:24.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:40:46.572 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:40:46.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:41:08.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:41:08.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:41:25.312 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:12:46.775 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.327 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:12:46.787 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.333 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:12:46.795 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.338 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:12:46.801 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.349 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:12:46.833 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.387 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:12:46.871 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.549 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:12:47.012 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.624 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:12:47.087 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:25.691 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:12:47.175 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:30.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:41:30.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:41:52.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:41:52.571+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:42:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:42:14.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:42:36.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:42:36.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:42:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:42:58.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:43:07.118 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:14:28.581 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.179 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:14:28.642 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.240 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:14:28.703 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.242 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:14:28.705 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.263 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:14:28.745 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.265 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:14:28.748 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.278 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:14:28.741 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.305 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:14:28.767 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:07.337 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:14:28.820 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:43:20.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:43:20.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:43:42.573 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:43:42.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:44:04.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:44:04.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:44:26.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:44:26.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:44:48.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:44:48.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:45:10.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:45:10.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:45:32.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:45:32.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:45:54.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:45:54.572+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:46:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:46:16.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:46:36.346 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:17:57.808 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.353 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:17:57.816 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.367 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:17:57.829 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.371 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:17:57.855 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.373 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:17:57.836 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.383 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:17:57.866 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.487 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:17:57.949 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.524 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:17:57.986 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:36.554 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:17:58.038 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:46:38.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:47:00.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:47:00.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:47:22.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:47:22.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:47:44.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:47:44.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:48:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:48:06.573+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:48:28.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:48:28.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:48:50.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:48:50.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:49:12.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:49:12.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:49:18.680 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:20:40.143 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.721 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:20:40.184 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.795 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:20:40.257 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.798 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:20:40.261 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.826 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:20:40.309 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.830 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:20:40.313 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.832 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:20:40.294 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.961 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:20:40.424 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:18.991 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:20:40.474 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:49:34.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:49:34.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:49:56.575 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:49:56.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:50:18.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:50:18.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:50:40.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:50:40.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:51:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:51:02.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:51:24.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:51:24.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:51:29.322 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:22:50.784 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.333 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:22:50.795 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.345 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:22:50.808 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.379 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:22:50.841 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.419 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:22:50.881 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.422 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:22:50.905 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.428 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:22:50.911 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.447 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:22:50.909 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:29.513 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:22:50.996 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.529 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:23:07.991 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.534 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:23:07.997 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.545 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:23:08.008 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.549 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:23:08.012 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.553 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:23:08.036 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.569 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:23:08.052 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.569 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:23:08.031 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:51:46.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:51:46.600 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:23:08.057 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:46.638 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:23:08.121 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:52:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:52:08.574+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:52:30.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:52:30.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:52:52.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:52:52.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:52:59.161 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:52:59.159+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c28042ab0 msgr2=0x7f5c28044ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:53:14.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:53:14.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:53:36.576 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:53:36.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:53:58.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:53:58.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:54:20.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:54:20.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:54:42.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:54:42.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:55:04.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:55:04.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:55:26.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:55:26.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:55:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:55:48.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:55:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:55:54.184+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c280347b0 msgr2=0x7f5c280248f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:56:10.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:56:10.575+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:56:32.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:56:32.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:56:54.577 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:56:54.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:56:58.235 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:28:19.698 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.244 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:28:19.706 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.253 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:28:19.716 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.255 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:28:19.738 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.257 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:28:19.720 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.259 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:28:19.742 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.450 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:28:19.912 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.452 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:28:19.915 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:58.469 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:28:19.952 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:57:16.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:57:16.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:57:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:57:38.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:58:00.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:58:00.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:58:22.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:58:22.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:58:44.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:58:44.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:59:06.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:59:06.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:59:28.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:59:28.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:59:50.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T17:59:50.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:59:50.953 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:31:12.415 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:50.962 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:31:12.425 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:50.997 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:31:12.459 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.001 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:31:12.461 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.003 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:31:12.485 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.055 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:31:12.538 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.093 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:31:12.556 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.119 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:31:12.581 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T17:59:51.161 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:31:12.644 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:12.578 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:00:12.576+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:00:34.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:00:34.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:00:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:00:56.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:01:18.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:01:18.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:01:40.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:01:40.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:02:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:02:02.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:02:06.640 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:33:28.102 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.654 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:33:28.117 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.655 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:33:28.117 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.663 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:33:28.125 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.672 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:33:28.154 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.702 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:33:28.185 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.770 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:33:28.232 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.806 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:33:28.269 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:06.896 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:33:28.378 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:24.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:02:24.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:02:46.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:02:46.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:03:08.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:03:08.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:03:30.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:03:30.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:03:52.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:03:52.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:04:14.579 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:04:14.577+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:04:36.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:04:36.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:04:58.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:04:58.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:05:20.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:05:20.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:05:42.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:05:42.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:06:04.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:06:04.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:06:26.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:06:26.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:06:48.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:06:48.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:07:10.580 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:07:10.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:07:20.611 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:38:42.073 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.616 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:38:42.078 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.630 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:38:42.092 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.634 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:38:42.097 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.648 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:38:42.130 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.653 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:38:42.136 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.826 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:38:42.288 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.900 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:38:42.382 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:20.960 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:38:42.422 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:32.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:07:32.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:07:54.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:07:54.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:08:12.704 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:39:34.166 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.732 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:39:34.194 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.741 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:39:34.203 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.756 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:39:34.218 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.789 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:39:34.271 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.791 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:39:34.274 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.824 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:39:34.286 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.833 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:39:34.295 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:12.857 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:39:34.339 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:16.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:08:16.578+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:08:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:08:38.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:09:00.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:09:00.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:09:22.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:09:22.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:09:44.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:09:44.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:10:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:10:06.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:10:28.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:10:28.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:10:50.581 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:10:50.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:11:12.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:11:12.579+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:11:34.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:11:34.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:11:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:11:56.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:12:18.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:12:18.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:12:30.747 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:43:52.209 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:30.754 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:43:52.216 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:30.763 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:43:52.225 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:30.767 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:43:52.229 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:30.784 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:43:52.266 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:30.799 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:43:52.281 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:30.924 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:43:52.386 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:30.934 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:43:52.396 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:31.103 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:43:52.585 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:40.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:12:40.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:13:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:13:02.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:13:24.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:13:24.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:13:46.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:13:46.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:14:08.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:14:08.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:14:30.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:14:30.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:14:52.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:14:52.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:15:14.582 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:15:14.580+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:15:36.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:15:36.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:15:58.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:15:58.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:16:20.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:16:20.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:16:34.430 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:47:55.892 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.433 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:47:55.895 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.484 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:47:55.946 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.486 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:47:55.949 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.519 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:47:56.001 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.528 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:47:56.011 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.587 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:47:56.049 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.620 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:47:56.082 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:34.668 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:47:56.150 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:16:42.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:16:42.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:17:04.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:17:04.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:17:26.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:17:26.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:17:42.928 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:49:04.390 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.931 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:49:04.393 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.940 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:49:04.403 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.947 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:49:04.409 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.950 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:49:04.432 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:42.969 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:49:04.451 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:43.005 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:49:04.467 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:43.011 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:49:04.473 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:43.080 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:49:04.562 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:48.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:17:48.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:18:10.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:18:10.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:18:32.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:18:32.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:18:54.583 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:18:54.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:19:16.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:19:16.581+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:19:38.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:19:38.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:20:00.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:20:00.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:20:22.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:20:22.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:20:24.457 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:20:24.455+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c28040690 msgr2=0x7f5c28042ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:20:44.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:20:44.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:21:04.441 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:21:04.440+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c280396d0 msgr2=0x7f5c28039ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:21:06.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:21:06.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:21:28.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:21:28.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:21:47.802 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:21:47.801+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5c2803d450 msgr2=0x7f5c2803d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:21:50.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:21:50.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:22:12.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:22:12.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:22:34.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:22:34.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:22:55.447 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:54:16.909 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.454 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:54:16.916 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.468 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:54:16.930 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.473 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:54:16.935 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.482 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:54:16.964 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.515 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:54:16.997 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.683 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:54:17.145 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.707 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:54:17.169 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:55.792 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:54:17.274 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:22:56.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:23:18.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:23:18.582+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:23:40.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:23:40.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:24:02.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:24:02.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:24:24.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:24:24.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:24:46.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:24:46.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:24:56.486 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:56:17.948 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.515 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:56:17.977 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.517 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:56:17.979 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.519 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:56:17.982 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.554 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:56:18.037 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.557 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:56:18.039 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.596 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:56:18.058 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.599 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:56:18.061 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:24:56.651 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:56:18.133 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:25:08.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:25:08.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:25:30.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:25:30.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:25:52.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:25:52.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:26:14.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:26:14.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:26:36.585 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:26:36.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:26:58.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:26:58.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:27:20.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:27:20.583+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:27:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:27:42.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:28:04.542 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:28:04.541+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c280093d0 msgr2=0x7f5c2800b7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:28:04.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:28:04.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:28:07.167 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:59:28.629 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.172 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:59:28.635 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.182 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:59:28.644 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.195 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:59:28.651 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.201 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:59:28.683 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.236 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:59:28.718 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.320 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:08:59:28.782 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.337 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:08:59:28.799 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:07.474 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:08:59:28.957 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:26.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:28:26.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:28:48.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:28:48.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:29:10.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:29:10.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:29:32.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:29:32.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:29:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:29:54.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:30:16.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:30:16.584+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:30:17.774 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:01:39.234 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:17.784 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:01:39.243 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:17.846 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:01:39.308 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:17.861 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:01:39.323 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:17.886 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:01:39.369 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:17.894 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:01:39.376 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:17.911 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:01:39.373 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:17.964 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:01:39.426 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:18.044 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:01:39.512 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:30:38.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:30:38.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:31:00.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:31:00.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:31:22.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:31:22.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:31:44.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:31:44.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:32:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:32:06.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:32:28.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:32:28.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:32:50.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:32:50.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:33:09.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:33:09.597+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c28040150 msgr2=0x7f5c2800fac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:33:12.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:33:12.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:33:17.718 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:04:39.180 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.722 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:04:39.184 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.732 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:04:39.194 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.740 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:04:39.201 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.745 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:04:39.227 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.751 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:04:39.233 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.870 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:04:39.332 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.901 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:04:39.383 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:17.926 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:04:39.388 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.375 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:04:40.837 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.376 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:04:40.839 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.388 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:04:40.850 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.571 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:04:41.053 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.573 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:04:41.056 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.590 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:04:41.072 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.785 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:04:41.246 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.785 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:04:41.248 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:19.795 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:04:41.257 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:33:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:33:34.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:33:54.567 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:33:54.566+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6810/10162,v1:172.21.15.33:6811/10162] conn(0x7f5c2800cc20 msgr2=0x7f5c2800f040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T18:33:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:33:56.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:34:18.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:34:18.585+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:34:40.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:34:40.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:35:02.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:35:02.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:35:24.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:35:24.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:35:46.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:35:46.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:36:08.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:36:08.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:36:30.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:36:30.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:36:52.588 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:36:52.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:37:14.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:37:14.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:37:36.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:37:36.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:37:58.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:37:58.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:38:20.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:38:20.586+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:38:29.676 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:09:51.122 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.679 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:09:51.124 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.680 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:09:51.137 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.680 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:09:51.161 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.681 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:09:51.141 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.687 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:09:51.169 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.871 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:09:51.333 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.986 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:09:51.468 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:29.997 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:09:51.459 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:38:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:38:42.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:39:04.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:39:04.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:39:26.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:39:26.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:39:48.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:39:48.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:40:10.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:40:10.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:40:32.589 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:40:32.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:40:54.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:40:54.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:41:16.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:41:16.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:41:38.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:41:38.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:41:40.294 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:13:01.756 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.300 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:13:01.759 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.340 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:13:01.802 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.342 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:13:01.804 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.375 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:13:01.857 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.383 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:13:01.865 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.391 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:13:01.853 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.451 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:13:01.913 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:40.497 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:13:01.976 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:54.698 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:41:54.697+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c280347b0 msgr2=0x7f5c280248f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:42:00.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:42:00.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:42:22.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:42:22.587+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:42:44.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:42:44.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:43:06.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:43:06.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:43:28.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:43:28.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:43:40.748 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:15:02.210 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.759 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:15:02.221 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.768 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:15:02.230 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.772 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:15:02.234 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.780 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:15:02.262 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.788 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:15:02.270 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.850 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:15:02.313 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.869 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:15:02.332 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:40.927 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:15:02.409 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:50.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:43:50.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:43:59.715 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:43:59.713+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c2800bd30 msgr2=0x7f5c28030ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:43:59.720 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:43:59.719+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c2800b8c0 msgr2=0x7f5c2800f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:44:12.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:44:12.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:44:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:44:34.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:44:56.590 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:44:56.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:45:14.692 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:45:14.690+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c28045000 msgr2=0x7f5c28040560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T18:45:18.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:45:18.588+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:45:39.708 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:45:39.707+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c2800bcd0 msgr2=0x7f5c28030ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:45:40.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:45:40.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:46:02.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:46:02.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:46:24.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:46:24.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:46:46.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:46:46.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:47:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:47:08.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:47:30.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:47:30.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:47:52.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:47:52.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:48:14.591 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:48:14.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:48:36.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:48:36.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:48:52.821 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:20:14.283 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:52.830 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:20:14.292 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:52.855 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:20:14.337 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:52.863 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:20:14.345 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:52.864 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:20:14.326 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:52.868 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:20:14.330 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:53.052 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:20:14.513 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:53.077 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:20:14.537 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:53.227 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:20:14.708 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:48:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:48:58.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:49:20.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:49:20.589+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:49:42.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:49:42.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:50:01.935 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:21:23.397 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:01.979 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:21:23.441 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:01.999 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:21:23.481 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:02.007 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:21:23.489 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:02.021 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:21:23.483 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:02.025 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:21:23.487 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:02.036 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:21:23.498 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:02.073 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:21:23.534 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:02.077 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:21:23.559 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:04.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:50:04.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:50:26.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:50:26.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:50:48.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:50:48.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:51:10.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:51:10.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:51:32.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:51:32.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:51:54.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:51:54.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:52:09.793 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:52:09.791+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6826/10160,v1:172.21.15.33:6827/10160] conn(0x7f5c28040150 msgr2=0x7f5c2800fac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:52:16.592 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:52:16.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:52:38.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:52:38.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:53:00.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:53:00.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:53:22.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:53:22.590+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:53:44.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:53:44.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:54:02.877 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:25:24.339 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:02.885 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:25:24.345 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:02.894 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:25:24.355 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:02.897 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:25:24.359 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:02.898 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:25:24.380 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:02.912 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:25:24.394 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:03.058 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:25:24.520 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:03.064 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:25:24.526 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:03.099 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:25:24.581 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:06.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:54:06.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:54:28.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:54:28.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:54:50.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:54:50.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:55:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:55:12.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:55:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:55:34.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:55:56.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:55:56.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:56:18.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:56:18.591+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:56:40.593 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:56:40.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:57:02.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:57:02.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:57:24.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:57:24.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:57:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:57:46.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:58:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:58:08.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:58:23.813 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:29:45.275 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:23.847 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:29:45.309 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:23.868 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:29:45.330 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:23.871 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:29:45.333 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:23.906 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:29:45.388 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:23.912 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:29:45.394 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:23.983 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:29:45.445 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:23.986 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:29:45.448 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:24.102 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:29:45.584 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:58:29.876+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c28045000 msgr2=0x7f5c28040560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T18:58:30.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:58:30.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:58:52.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:58:52.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:59:14.054 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:30:35.516 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.065 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:30:35.527 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.075 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:30:35.537 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.079 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:30:35.541 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.090 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:30:35.572 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.109 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:30:35.591 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.136 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:30:35.598 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.144 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:30:35.606 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.208 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:30:35.690 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:14.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:59:14.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:59:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:59:36.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T18:59:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T18:59:58.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:00:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:00:20.592+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:00:42.594 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:00:42.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:01:04.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:01:04.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:01:26.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:01:26.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:01:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:01:48.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:02:10.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:02:10.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:02:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:02:32.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:02:54.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:02:54.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:03:16.595 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:03:16.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:03:38.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:03:38.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:04:00.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:04:00.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:04:09.935 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:04:09.932+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c2800bcd0 msgr2=0x7f5c28030ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T19:04:22.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:04:22.593+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:04:25.063 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:35:46.525 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.077 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:35:46.539 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.081 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:35:46.543 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.086 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:35:46.548 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.095 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:35:46.577 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.125 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:35:46.603 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.389 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:35:46.849 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.401 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:35:46.861 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:25.401 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:35:46.878 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:04:44.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:04:44.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:05:06.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:05:06.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:05:28.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:05:28.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:05:50.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:05:50.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:06:12.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:06:12.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:06:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:06:34.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:06:45.502 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:38:06.964 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.508 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:38:06.971 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.579 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:38:07.041 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.601 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:38:07.063 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.603 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:38:07.086 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.606 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:38:07.089 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.609 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:38:07.072 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.692 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:38:07.153 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:45.723 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:38:07.206 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:56.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:06:56.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:07:18.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:07:18.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:07:40.596 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:07:40.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:08:02.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:08:02.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:08:24.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:08:24.594+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:08:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:08:46.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:09:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:09:08.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:09:30.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:09:30.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:09:36.225 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:40:57.683 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.228 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:40:57.691 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.241 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:40:57.703 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.245 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:40:57.707 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.256 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:40:57.738 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.284 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:40:57.766 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.352 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:40:57.814 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.384 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:40:57.846 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:36.506 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:40:57.983 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:09:52.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:09:52.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:10:14.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:10:14.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:10:36.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:10:36.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:10:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:10:58.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:11:16.727 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:42:38.189 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.778 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:42:38.240 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.799 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:42:38.261 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.804 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:42:38.266 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.845 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:42:38.327 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.848 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:42:38.330 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.870 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:42:38.333 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.891 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:42:38.353 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:16.989 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:42:38.472 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:11:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:11:20.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:11:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:11:42.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:12:04.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:12:04.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:12:26.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:12:26.595+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:12:48.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:12:48.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:13:10.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:13:10.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:13:32.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:13:32.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:13:47.858 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:13:47.857+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6810/10132,v1:172.21.15.186:6811/10132] conn(0x7f5c28045000 msgr2=0x7f5c28040560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T19:13:54.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:13:54.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:14:10.887 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:14:10.886+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3006146966 >> [v2:172.21.15.33:6810/10162,v1:172.21.15.33:6811/10162] conn(0x7f5b88024550 msgr2=0x7f5b88023ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T19:14:16.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:14:16.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:14:38.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:14:38.596+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:14:47.585 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:46:09.047 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.589 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:46:09.051 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.599 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:46:09.061 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.603 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:46:09.065 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.616 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:46:09.098 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.631 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:46:09.113 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.730 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:46:09.191 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.780 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:46:09.242 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.922 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:46:09.404 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:00.598 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:15:00.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:15:07.606 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:46:29.068 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.608 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:46:29.070 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.677 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:46:29.139 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.679 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:46:29.141 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.692 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:46:29.154 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.698 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:46:29.181 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.701 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:46:29.183 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.721 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:46:29.183 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:07.739 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:46:29.221 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:15:22.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:15:22.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:15:44.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:15:44.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:16:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:16:06.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:16:28.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:16:28.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:16:50.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:16:50.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:17:12.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:17:12.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:17:34.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:17:34.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:17:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:17:56.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:18:18.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:18:18.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:18:40.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:18:40.597+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:19:02.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:19:02.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:19:10.104 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:19:10.103+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c2800bcd0 msgr2=0x7f5c28030ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T19:19:24.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:19:24.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:19:46.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:19:46.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:19:59.777 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:51:21.234 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:59.782 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:51:21.244 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:59.790 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:51:21.252 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:59.798 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:51:21.260 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:59.805 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:51:21.287 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:59.816 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:51:21.298 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:00.055 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:51:21.517 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:00.072 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:51:21.534 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:00.124 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:51:21.606 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:08.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:20:08.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:20:30.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:20:30.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:20:52.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:20:52.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:21:14.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:21:14.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:21:36.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:21:36.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:21:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:21:58.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:22:20.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:22:20.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:22:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:22:42.598+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:23:04.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:23:04.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:23:26.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:23:26.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:23:34.334 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:54:55.796 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.392 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:54:55.854 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.400 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:54:55.862 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.408 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:54:55.870 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.423 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:54:55.906 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.427 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:54:55.909 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.491 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:54:55.953 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.513 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:54:55.975 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:34.588 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:54:56.070 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:23:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:23:48.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:24:10.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:24:10.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:24:32.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:24:32.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:24:54.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:24:54.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:24:55.166 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:24:55.165+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6826/10160,v1:172.21.15.33:6827/10160] conn(0x7f5c28031810 msgr2=0x7f5c28007960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T19:25:09.907 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:56:31.369 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:09.911 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:56:31.373 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:09.933 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:56:31.395 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:09.940 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:56:31.402 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:09.944 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:56:31.426 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:09.956 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:56:31.438 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:10.005 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:09:56:31.466 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:10.041 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:09:56:31.503 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:10.136 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:09:56:31.619 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:25:16.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:25:16.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:25:38.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:25:38.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:26:00.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:26:00.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:26:22.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:26:22.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:26:44.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:26:44.599+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:27:06.601 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:27:06.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:27:28.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:27:28.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:27:50.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:27:50.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:28:12.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:28:12.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:28:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:28:34.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:28:56.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:28:56.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:29:18.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:29:18.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:29:40.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:29:40.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:30:02.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:30:02.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:30:20.761 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:01:42.223 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:20.770 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:01:42.232 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:20.786 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:01:42.248 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:20.789 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:01:42.251 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:20.798 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:01:42.280 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:20.805 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:01:42.287 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:20.978 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:01:42.440 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:21.096 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:01:42.558 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:21.121 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:01:42.603 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:24.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:30:24.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:30:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:30:46.600+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:31:08.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:31:08.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:31:30.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:31:30.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:31:52.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:31:52.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:31:56.067 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:03:17.529 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.097 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:03:17.558 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.112 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:03:17.575 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.114 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:03:17.576 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.164 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:03:17.646 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.172 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:03:17.654 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.173 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:03:17.635 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.176 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:03:17.638 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:31:56.288 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:03:17.765 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:32:14.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:32:14.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:32:36.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:32:36.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:32:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:32:58.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:33:20.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:33:20.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:33:42.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:33:42.601+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:34:04.603 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:34:04.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:34:26.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:34:26.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:34:48.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:34:48.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:35:10.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:35:10.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:35:32.101 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:06:53.563 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.113 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:06:53.575 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.131 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:06:53.593 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.134 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:06:53.597 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.137 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:06:53.620 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.163 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:06:53.645 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.320 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:06:53.782 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.326 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:06:53.789 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.418 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:06:53.896 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:32.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:35:32.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:35:54.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:35:54.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:36:16.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:36:16.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:36:38.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:36:38.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:37:00.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:37:00.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:37:22.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:37:22.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:37:44.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:37:44.602+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:38:06.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:38:06.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:38:28.604 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:38:28.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:38:50.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:38:50.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:39:12.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:39:12.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:39:15.323 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:39:15.322+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c280347b0 msgr2=0x7f5c280248f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T19:39:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:39:34.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:39:56.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:39:56.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:40:17.857 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:11:39.318 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:17.863 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:11:39.325 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:17.933 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:11:39.391 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:17.933 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:11:39.393 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:17.956 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:11:39.436 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:17.959 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:11:39.441 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:18.085 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:11:39.547 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:18.113 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:11:39.575 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:18.200 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:11:39.682 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:18.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:40:18.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:40:40.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:40:40.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:40:44.188 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:12:05.649 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.195 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:12:05.657 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.197 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:12:05.660 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.203 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:12:05.665 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.212 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:12:05.694 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.230 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:12:05.706 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.236 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:12:05.699 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.249 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:12:05.708 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:40:44.297 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:12:05.779 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:41:02.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:41:02.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:41:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:41:24.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:41:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:41:46.603+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:42:08.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:42:08.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:42:30.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:42:30.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:42:52.605 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:42:52.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:43:14.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:43:14.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:43:36.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:43:36.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:43:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:43:58.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:44:20.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:44:20.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:44:42.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:44:42.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:45:04.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:45:04.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:45:26.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:45:26.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:45:48.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:45:48.604+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:45:57.150 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:17:18.610 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.161 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:17:18.616 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.166 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:17:18.628 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.174 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:17:18.655 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.174 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:17:18.634 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.185 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:17:18.668 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.357 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:17:18.819 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.425 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:17:18.888 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:45:57.478 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:17:18.961 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:10.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:46:10.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:46:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:46:28.749+0000 7f5c474b0700 0 -- 172.21.15.33:0/3006146966 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5b88024550 msgr2=0x7f5b88041eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T19:46:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:46:28.951+0000 7f5c474b0700 0 --2- 172.21.15.33:0/3006146966 >> [v2:172.21.15.33:3301/0,v1:172.21.15.33:6790/0] conn(0x7f5b88024550 0x7f5b88041eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T19:46:32.606 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:46:32.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:46:54.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:46:54.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:47:10.374 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:47:10.373+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c2800bcd0 msgr2=0x7f5c28030ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T19:47:16.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:47:16.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:47:38.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:47:38.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:48:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:48:00.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:48:22.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:48:22.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:48:39.621 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:20:01.082 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.628 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:20:01.091 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.689 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:20:01.152 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.692 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:20:01.154 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.715 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:20:01.177 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.723 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:20:01.206 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.727 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:20:01.210 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.779 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:20:01.241 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:39.857 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:20:01.339 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:48:44.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:48:44.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:49:06.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:49:06.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:49:28.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:49:28.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:49:50.607 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:49:50.605+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:50:12.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:50:12.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:50:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:50:34.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:50:56.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:50:56.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:51:07.244 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:22:28.706 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.254 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:22:28.716 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.263 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:22:28.726 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.267 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:22:28.748 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.267 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:22:28.729 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.271 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:22:28.753 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.379 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:22:28.841 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.394 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:22:28.856 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:07.425 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:22:28.907 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:51:18.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:51:18.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:51:40.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:51:40.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:52:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:52:02.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:52:20.676 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:23:42.138 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.712 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:23:42.174 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.740 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:23:42.222 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.768 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:23:42.228 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.777 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:23:42.238 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.785 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:23:42.246 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.808 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:23:42.290 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.882 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:23:42.344 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:20.968 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:23:42.451 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:52:24.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:52:46.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:52:46.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:53:08.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:53:08.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:53:25.443 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:53:25.442+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c2800bcd0 msgr2=0x7f5c28030ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T19:53:30.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:53:30.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:53:52.608 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:53:52.606+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:54:14.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:54:14.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:54:36.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:54:36.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:54:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:54:58.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:55:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:55:20.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:55:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:55:42.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:56:04.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:56:04.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:56:18.835 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:27:40.297 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:18.849 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:27:40.311 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:18.855 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:27:40.317 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:18.862 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:27:40.325 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:18.869 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:27:40.351 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:18.878 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:27:40.360 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:19.011 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:27:40.473 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:19.013 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:27:40.476 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:19.055 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:27:40.537 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:56:26.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:56:26.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:56:48.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:56:48.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:57:01.583 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:28:23.045 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.592 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:28:23.055 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.623 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:28:23.086 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.625 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:28:23.088 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.647 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:28:23.110 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.657 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:28:23.139 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.663 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:28:23.145 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.668 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:28:23.130 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:01.710 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:28:23.192 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T19:57:10.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:57:10.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:57:32.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:57:32.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:57:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:57:54.607+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:58:16.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:58:16.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:58:38.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:58:38.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:59:00.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:59:00.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:59:22.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:59:22.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T19:59:44.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T19:59:44.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:00:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:00:06.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:00:28.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:00:28.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:00:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:00:40.535+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6826/10160,v1:172.21.15.33:6827/10160] conn(0x7f5c28007ea0 msgr2=0x7f5c2800b380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:00:50.610 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:00:50.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:01:12.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:01:12.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:01:28.984 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:32:50.446 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:28.993 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:32:50.455 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:29.008 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:32:50.490 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:29.009 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:32:50.471 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:29.018 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:32:50.500 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:29.023 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:32:50.485 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:29.191 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:32:50.673 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:29.292 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:32:50.753 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:29.310 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:32:50.772 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:34.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:01:34.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:01:56.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:01:56.608+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:02:18.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:02:18.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:02:40.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:02:40.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:03:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:03:02.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:03:24.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:03:24.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:03:46.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:03:46.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:04:08.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:04:08.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:04:30.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:04:30.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:04:52.611 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:04:52.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:05:14.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:05:14.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:05:23.413 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:36:44.875 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.429 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:36:44.885 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.448 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:36:44.907 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.476 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:36:44.938 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.509 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:36:44.991 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.512 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:36:44.994 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.525 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:36:44.986 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.617 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:36:45.079 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:23.628 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:36:45.110 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:05:36.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:05:36.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:05:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:05:58.609+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:06:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:06:20.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:06:41.092 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:38:02.553 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.099 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:38:02.561 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.110 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:38:02.572 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.113 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:38:02.575 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.129 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:38:02.611 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.149 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:38:02.631 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.178 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:38:02.640 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.198 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:38:02.660 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:41.310 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:38:02.792 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:06:42.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:06:42.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:07:04.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:07:04.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:07:26.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:07:26.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:07:48.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:07:48.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:08:10.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:08:10.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:08:32.612 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:08:32.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:08:54.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:08:54.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:09:16.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:09:16.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:09:38.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:09:38.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:10:00.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:10:00.610+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:10:22.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:10:22.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:10:44.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:10:44.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:11:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:11:06.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:11:28.613 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:11:28.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:11:50.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:11:50.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:11:53.269 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:43:14.730 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.275 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:43:14.737 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.285 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:43:14.747 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.290 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:43:14.752 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.299 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:43:14.781 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.309 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:43:14.791 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.497 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:43:14.956 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.507 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:43:14.968 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:53.587 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:43:15.069 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:12:12.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:12:12.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:12:34.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:12:34.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:12:56.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:12:56.611+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:13:18.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:13:18.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:13:40.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:13:40.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:13:50.235 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:45:11.696 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.247 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:45:11.708 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.279 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:45:11.740 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.281 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:45:11.743 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.313 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:45:11.774 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.343 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:45:11.825 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.345 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:45:11.827 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.379 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:45:11.840 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:13:50.436 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:45:11.918 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:02.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:14:02.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:14:24.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:14:24.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:14:46.614 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:14:46.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:15:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:15:08.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:15:25.720 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:15:25.719+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c28007ea0 msgr2=0x7f5c2800b380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:15:30.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:15:30.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:15:52.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:15:52.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:16:14.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:16:14.612+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:16:36.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:16:36.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:16:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:16:58.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:17:05.528 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:48:26.990 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.535 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:48:26.996 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.544 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:48:27.006 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.548 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:48:27.010 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.567 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:48:27.049 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.577 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:48:27.058 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.696 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:48:27.158 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.699 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:48:27.161 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:05.704 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:17:05.702+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c28033e00 msgr2=0x7f5c28031d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T20:17:05.775 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:48:27.257 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:20.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:17:20.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:17:42.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:17:42.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:18:04.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:18:04.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:18:26.615 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:18:26.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:18:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:18:40.720+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c280347b0 msgr2=0x7f5c280248f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T20:18:48.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:18:48.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:18:51.599 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:18:51.598+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3006146966 >> [v2:172.21.15.33:6810/10162,v1:172.21.15.33:6811/10162] conn(0x7f5b88009ad0 msgr2=0x7f5b88048d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T20:19:10.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:19:10.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:19:32.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:19:32.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:19:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:19:54.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:20:16.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:20:16.613+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:20:38.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:20:38.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:21:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:21:00.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:21:22.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:21:22.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:21:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:21:44.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:22:06.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:22:06.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:22:12.165 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:53:33.627 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.174 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:53:33.635 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.199 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:53:33.660 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.221 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:53:33.683 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.263 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:53:33.745 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.266 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:53:33.748 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.349 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:53:33.811 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.459 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:53:33.921 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:12.537 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:53:34.019 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.815 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:53:37.276 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.821 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:53:37.283 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.832 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:53:37.294 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.839 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:53:37.301 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.840 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:53:37.322 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.844 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:53:37.306 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.856 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:53:37.333 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.872 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:53:37.334 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:15.921 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:53:37.404 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:28.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:22:28.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:22:50.616 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:22:50.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:23:12.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:23:12.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:23:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:23:34.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:23:56.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:23:56.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:24:18.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:24:18.614+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:24:40.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:24:40.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:25:02.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:25:02.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:25:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:25:05.839+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c28007ea0 msgr2=0x7f5c2800b380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:25:24.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:25:24.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:25:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:25:46.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:26:08.617 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:26:08.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:26:30.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:26:30.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:26:52.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:26:52.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:27:14.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:27:14.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:27:28.259 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:58:49.721 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.272 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:58:49.734 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.280 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:58:49.742 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.285 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:58:49.747 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.294 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:58:49.776 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.308 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:58:49.790 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.481 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:10:58:49.935 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.481 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:10:58:49.940 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:28.557 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:10:58:50.039 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:36.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:27:36.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:27:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:27:58.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:28:20.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:28:20.615+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:28:42.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:28:42.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:29:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:29:04.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:29:26.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:29:26.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:29:48.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:29:48.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:30:10.618 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:30:10.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:30:32.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:30:32.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:30:33.971 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:01:55.433 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:33.977 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:01:55.439 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:34.028 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:01:55.490 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:34.037 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:01:55.493 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:34.046 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:01:55.528 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:34.061 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:01:55.543 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:34.091 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:01:55.553 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:34.128 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:01:55.589 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:34.212 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:01:55.692 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:54.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:30:54.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:31:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:31:16.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:31:38.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:31:38.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:31:40.878 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:31:40.876+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c28007ea0 msgr2=0x7f5c2800b380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:32:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:32:00.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:32:22.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:32:22.616+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:32:40.515 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:04:01.977 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.522 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:04:01.984 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.527 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:04:01.989 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.532 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:04:01.994 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.548 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:04:02.030 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.566 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:04:02.048 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.626 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:04:02.088 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.633 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:04:02.095 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:40.706 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:04:02.189 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:32:44.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:32:44.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:33:06.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:33:06.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:33:28.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:33:28.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:33:29.726 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:04:51.185 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:29.761 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:04:51.223 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:29.782 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:04:51.244 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:29.789 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:04:51.251 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:29.858 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:04:51.341 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:29.861 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:04:51.344 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:29.863 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:04:51.325 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:29.883 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:04:51.345 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:30.025 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:04:51.507 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:33:50.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:33:50.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:34:12.619 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:34:12.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:34:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:34:34.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:34:56.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:34:56.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:35:18.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:35:18.617+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:35:40.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:35:40.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:36:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:36:02.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:36:24.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:36:24.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:36:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:36:46.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:37:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:37:08.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:37:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:37:30.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:37:52.438 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:09:13.900 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.443 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:09:13.905 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.462 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:09:13.924 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.463 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:09:13.944 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.469 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:09:13.952 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.470 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:09:13.932 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:37:52.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:37:52.633 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:09:14.095 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.685 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:09:14.146 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:37:52.789 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:09:14.271 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:14.620 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:38:14.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:38:36.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:38:36.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:38:55.956 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:10:17.418 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.007 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:10:17.469 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.018 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:10:17.478 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.018 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:10:17.480 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.044 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:10:17.527 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.051 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:10:17.533 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.078 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:10:17.540 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.080 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:10:17.542 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:56.109 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:10:17.592 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:38:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:38:58.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:39:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:39:20.618+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:39:42.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:39:42.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:40:04.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:40:04.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:40:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:40:26.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:40:48.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:40:48.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:41:10.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:41:10.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:41:25.972 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:41:25.970+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c2800bcd0 msgr2=0x7f5c28030ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T20:41:32.621 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:41:32.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:41:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:41:54.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:42:16.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:42:16.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:42:38.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:42:38.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:43:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:43:00.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:43:02.918 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:14:24.380 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:02.929 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:14:24.391 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:02.936 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:14:24.398 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:02.950 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:14:24.411 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:02.952 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:14:24.435 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:02.962 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:14:24.445 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:03.108 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:14:24.570 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:03.116 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:14:24.578 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:03.240 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:14:24.723 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:22.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:43:22.619+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:43:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:43:44.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:44:06.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:44:06.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:44:28.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:44:28.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:44:50.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:44:50.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:45:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:45:12.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:45:34.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:45:34.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:45:56.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:45:56.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:46:18.622 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:46:18.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:46:40.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:46:40.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:47:02.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:47:02.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:47:17.861 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:18:39.323 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:17.924 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:18:39.367 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:17.924 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:18:39.369 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:17.925 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:18:39.379 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:17.942 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:18:39.424 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:17.944 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:18:39.426 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:18.016 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:18:39.478 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:18.059 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:18:39.521 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:18.074 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:18:39.555 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:24.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:47:24.620+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:47:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:47:46.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:48:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:48:08.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:48:14.790 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:19:36.252 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:14.798 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:19:36.260 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:14.814 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:19:36.275 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:14.820 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:19:36.282 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:14.823 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:19:36.304 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:14.835 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:19:36.317 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:14.886 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:19:36.348 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:14.892 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:19:36.354 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:15.027 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:19:36.509 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:48:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:48:30.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:48:52.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:48:52.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:49:11.103 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:49:11.101+0000 7f5c474b0700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6824/10131,v1:172.21.15.186:6825/10131] conn(0x7f5c280347b0 msgr2=0x7f5c280248f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:49:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:49:14.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:49:26.072 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:49:26.069+0000 7f5c47cb1700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6802/10161,v1:172.21.15.33:6803/10161] conn(0x7f5c2800b8c0 msgr2=0x7f5c2800f580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:49:36.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:49:36.621+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:49:58.623 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:49:58.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:50:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:50:20.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:50:42.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:50:42.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:51:04.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:51:04.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:51:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:51:26.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:51:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:51:48.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:52:10.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:52:10.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:52:32.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:52:32.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:52:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:52:54.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:53:16.624 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:53:16.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:53:25.252 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:24:46.714 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.268 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:24:46.730 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.275 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:24:46.737 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.282 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:24:46.744 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.286 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:24:46.765 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.293 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:24:46.775 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.485 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:24:46.947 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.582 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:24:47.065 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:25.595 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:24:47.057 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:53:36.143 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:53:36.140+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.186:6808/10130,v1:172.21.15.186:6809/10130] conn(0x7f5c28033e00 msgr2=0x7f5c28031d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:53:38.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:53:38.622+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:54:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:54:00.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:54:22.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:54:22.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:54:44.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:54:44.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:55:06.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:55:06.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:55:28.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:55:28.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:55:39.637 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:27:01.098 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.668 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:27:01.129 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.701 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:27:01.163 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.706 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:27:01.188 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.713 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:27:01.175 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.741 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:27:01.223 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.779 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:27:01.241 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.788 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:27:01.250 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:39.836 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:27:01.318 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:55:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:55:50.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:56:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:56:12.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:56:34.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:56:34.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:56:56.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:56:56.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:57:18.625 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:57:18.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:57:40.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:57:40.623+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:58:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:58:02.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:58:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:58:24.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:58:36.429 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:29:57.891 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.433 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:29:57.896 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.462 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:29:57.944 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.470 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:29:57.953 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.490 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:29:57.942 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.490 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:29:57.946 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.557 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:29:58.019 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.622 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:29:58.084 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:36.660 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:29:58.142 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T20:58:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:58:46.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:59:08.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:59:08.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:59:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:59:30.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T20:59:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:59:46.185+0000 7f5c3ecaf700 0 -- 172.21.15.33:0/3563239545 >> [v2:172.21.15.33:6826/10160,v1:172.21.15.33:6827/10160] conn(0x7f5c28040150 msgr2=0x7f5c2800fac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T20:59:52.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T20:59:52.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:00:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:00:14.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:00:36.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:00:36.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:00:58.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:00:58.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:01:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:01:20.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:01:42.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:01:42.624+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:02:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:02:04.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:02:26.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:02:26.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:02:48.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:02:48.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:03:10.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:03:10.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:03:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:03:32.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:03:46.576 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:35:08.038 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.590 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:35:08.052 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.600 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:35:08.062 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.606 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:35:08.068 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.607 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:35:08.089 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.625 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:35:08.108 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.802 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:35:08.264 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.898 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:35:08.360 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:46.936 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:35:08.418 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T21:03:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:03:54.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:04:01.433 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:35:22.894 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.462 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:35:22.924 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.471 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:35:22.933 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.494 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:35:22.956 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.502 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:35:22.985 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.503 INFO:tasks.ceph.mon.a.smithi033.stderr:==00:11:35:22.966 9134== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.517 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:35:23.000 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.524 INFO:tasks.ceph.mon.c.smithi033.stderr:==00:11:35:22.986 9133== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:01.548 INFO:tasks.ceph.mon.b.smithi186.stderr:==00:11:35:23.030 8825== Warning: unimplemented fcntl command: 1036 2019-12-09T21:04:16.627 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:04:16.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:04:38.633 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:04:38.625+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:05:00.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:05:00.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:05:22.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:05:22.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:05:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:05:44.626+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:06:06.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:06:06.627+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:06:28.628 INFO:teuthology.orchestra.run.smithi033.stderr:2019-12-09T21:06:28.627+0000 7f5c17fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T21:06:50.440 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-12-09T21:06:50.441 DEBUG:teuthology.task.console_log:Killing console logger for smithi186 2019-12-09T21:06:50.441 DEBUG:teuthology.task.console_log:Killing console logger for smithi033 2019-12-09T21:06:50.441 DEBUG:teuthology.task.console_log:Killing console logger for smithi186 2019-12-09T21:06:50.442 DEBUG:teuthology.task.console_log:Killing console logger for smithi033 2019-12-09T21:06:50.442 DEBUG:teuthology.exit:Finished running handlers